is there any way to not have redshift-destination always write varchar(6565535) its bad for redshift query performance - why does it not mirror the length of the source where possible

Thinking completed Redshift Destination and VARCHAR(65535) Unfortunately, this is a known limitation of the Airbyte Redshift destination — there is currently no way to configure it to mirror the source VARCHAR length. Here's why: Why it always uses VARCHAR(65535) Airbyte uses a unified internal type system that maps source types to Airbyte types, and then from Airbyte types to destination types. The Redshift destination maps the Airbyte

STRING

type to

VARCHAR(65535)

unconditionally, regardless of the original column length in the source. [Data type map] This is also hardcoded in the connector's data type enum: [RedshiftDataType.kt]

VARCHAR("varchar(65535)"),

Additionally, when the connector reads back the schema from Redshift's

information_schema

, it normalizes all

character varying

columns back to

varchar(65535)

, ignoring the actual declared length: [RedshiftAirbyteClient.kt]

"character varying" -> "varchar(65535)"

Why this design choice exists The rationale is likely safety and simplicity — Airbyte's type system abstracts away source-specific precision/length details to avoid truncation errors when syncing data. If a source column is declared

VARCHAR(500)

but a row somehow contains more data, using a smaller destination type could cause failures. What you can do The knowledge sources do not describe any configuration option to change this behavior. Your practical options are limited to: 1. Post-processing: After sync,

ALTER COLUMN

the specific columns you care about to a smaller

VARCHAR

size (if you know the data fits). 2. Create views: Build views on top of the Airbyte-synced tables that

CAST

columns to smaller types for query performance. 3. Feature request: Open or upvote a GitHub issue on the Airbyte repository requesting source-length-aware VARCHAR sizing for the Redshift destination. Your concern about query performance is valid — Redshift does allocate memory based on declared column widths in some operations, so

VARCHAR(65535)

across the board can impact sort/hash performance. Unfortunately, the connector does not currently support mirroring source lengths. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.