<@U04S8PLJWMT> I upgraded a postgres destination c...
# replication-ask-ai
h
@kapa.ai I upgraded a postgres destination connector to version 3 form v2. Now in my warehouse i am facing the issue where ince the Dalgo Postgres Destination Connector upgrade, our pipeline and connectors have been failing. The cause is how the upgraded connector names columns. For long column names (mostly form question fields), it now cuts the name short and adds a number at the end. For example, in the CSAT_Form_Report table: Attach check and tell me
k
Thinking completed Column Name Truncation After Postgres Destination v3 Upgrade Yes, I can confirm what you're seeing. Looking at your attached screenshot, columns like: •
How_useful_did_you_find_the_orientation_context_settin_33189303
•
What_type_of_volunteer_activity_would_you_like_to_part_54944823
•
Please_share_any_feedback_or_suggestions_regarding_the_45439332
These are long column names that have been truncated to 63 characters with a hash suffix appended. This is expected behavior from the connector. Why This Happens PostgreSQL restricts all identifiers to a maximum of 63 bytes. The Postgres destination connector enforces this limit. When a column name exceeds 63 characters, the connector: [Naming Limitations] 1. Takes the first 54 characters of the name 2. Appends
_
+ an 8-character hash (derived from
hashCode()
) to avoid collisions This is implemented in the connector's naming utility: [PostgresNamingUtils]
Copy code
if (transformed.length > 63) {
    val hash = transformed.hashCode().toString().takeLast(8)
    transformed = transformed.take(54) + "_" + hash
}
So for example,
How_useful_did_you_find_the_orientation_context_setting_...
gets truncated to 54 chars +
_
+ 8-digit hash =
How_useful_did_you_find_the_orientation_context_settin_33189303
. What Changed from v2 to v3 In v2, the connector used a different naming/truncation approach. The upgrade to v3 (Direct Load architecture) changed how column names are transformed, which means column names that were previously handled differently are now being renamed, breaking downstream pipelines that referenced the old column names. [Postgres Migration Guide] What You Can Do 1. Update downstream queries/dbt models to reference the new truncated column names (e.g.,
How_useful_did_you_find_the_orientation_context_settin_33189303
instead of the old name). 2. Check for collisions: If two long column names truncate to the same 54-character prefix, the hash suffix differentiates them — make sure your downstream logic accounts for this. 3. Review the migration guide for other breaking changes introduced in v3, such as the removal of
_airbyte_loaded_at
and the new
_airbyte_meta
/
_airbyte_generation_id
columns (visible in your screenshot). [Postgres Migration Guide] Unfortunately, there is no connector-level setting to bypass the 63-character PostgreSQL identifier limit — this is a hard constraint imposed by PostgreSQL itself. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
h
@kapa.ai can you confirm that the truncation used to happen in v2 also and not only v3 ?
k
Thinking completed I'm sorry, but the knowledge sources do not contain enough information to confirm the exact truncation behavior in Postgres destination v2 versus v3. What the knowledge sources do confirm is: • Postgres has always had a 63-character identifier limit, and the connector has always needed to handle this. [Naming Limitations] • The v2 upgrade (Destinations V2) introduced Typing & Deduping and changed how column names were handled (e.g., preserving mixed case, using quoted identifiers). [Destinations V2 Differences] • The legacy normalization (pre-v2) also performed truncation, as seen in forum logs showing names being truncated (e.g.,
us_p0_json_18_properties_view_columns_format
truncated to 43 chars). [Forum Post] So truncation did exist before v3, but the knowledge sources do not explicitly document whether the specific truncation algorithm (54 chars +
_
+ 8-digit hash) was the same in v2 or different. The breaking change you're experiencing is most likely due to a change in the truncation logic or naming convention between v2 and v3, not the introduction of truncation itself. To get a definitive answer, I'd recommend checking the Airbyte GitHub changelog for the Postgres destination connector directly. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.