<@U04S8PLJWMT> I’m trying to sync tables with Chin...
# replication-ask-ai
w
@kapa.ai I’m trying to sync tables with Chinese column names, but the auto-normalization is renaming them, which I want to avoid. I need to sync them as Raw Data or keep the original naming. Currently, I don’t see the option in the Connection Settings or Stream configuration. Is there a workaround or a specific setting I should look for? Any pointers would be appreciated! 🚀
k
This is a known issue with Airbyte's name transformation behavior. Here's what the knowledge sources tell us: What's Happening Airbyte's destination connectors apply a
StandardNameTransformer
(or destination-specific variants) that calls
toAlphanumericAndUnderscore
, which replaces non-ASCII characters — including Chinese characters — with underscores. This happens at the destination level during the typing & deduping process. [normalization issue] Current Workarounds / Options 1. Raw Data (no normalization): In older Airbyte versions, you could select "Raw data (JSON)" as the sync mode, which would write data to `_airbyte_raw_`stream_name tables with an
_airbyte_data
JSON blob column — bypassing column name transformation entirely. However, this means your Chinese column names would be preserved inside the JSON blob, not as individual columns. [normalization overview] 2. Typing & Deduping (Destinations V2): In the current Airbyte V2, Basic Normalization has been deprecated in favor of Typing & Deduping. Unfortunately, the column renaming issue persists in this pipeline as well. [normalization issue] 3. File a GitHub Issue: The knowledge sources suggest that the best path forward for non-ASCII/special character column name preservation is to open a GitHub issue so the Airbyte team can adjust the
StandardNameTransformer
to support a wider set of characters. This was the recommendation made for a similar Kanji character issue with Redshift. [normalization issue] Important Caveat The knowledge sources do not explicitly describe a UI setting or connector-level configuration that allows you to fully bypass column name normalization while still getting individual columns in the destination. The only confirmed bypass is syncing as raw JSON (which keeps names in the blob, not as columns). I'd recommend: • Checking the Airbyte GitHub issues for an existing report on Chinese/CJK column name preservation. • Opening a new issue if one doesn't exist, referencing the
toAlphanumericAndUnderscore
behavior in
Names.kt
as the root cause. [normalization issue] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
w
@kapa.ai what if I want to keep the Chinese character in columns?
k
Based on the knowledge sources, there is currently no supported configuration or UI setting in Airbyte that allows you to preserve Chinese (CJK) characters as actual column names in the destination table. Here's why: • Airbyte's
StandardNameTransformer
calls
toAlphanumericAndUnderscore
, which strips non-ASCII characters (including Chinese characters) and replaces them with underscores. This happens at the destination level and is not configurable via the UI. [normalization issue] • With Destinations V2 (Typing & Deduping), this transformation still applies, and Basic Normalization (which could previously be toggled off) has been deprecated. [normalization issue] The Only Partial Workaround Syncing as Raw Data (no normalization) will write your data as a JSON blob in the
_airbyte_data
column of the `_airbyte_raw_`stream_name table. In this case, the Chinese column names are preserved inside the JSON blob, but they won't appear as individual columns in the destination. [normalization overview] Recommended Action The knowledge sources explicitly recommend filing a GitHub issue with the Airbyte team to request that the
StandardNameTransformer
be updated to support a wider character set (including CJK characters), since destinations like Redshift and Postgres technically support quoted identifiers that can contain such characters. [normalization issue] In short, keeping Chinese characters as actual destination column names is not currently possible through any available Airbyte setting. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
w
@kapa.ai but airbyte 0.32 support
k
The knowledge sources do not contain any information about Airbyte version 0.32 or its specific behavior regarding Chinese/CJK column name preservation. I cannot confirm or deny what that specific version supported based on the available documentation. What I can say is that the knowledge sources only reference the normalization issue in the context of more recent versions (e.g., 0.40.x and Destinations V2), and the recommended path for preserving non-ASCII column names remains filing a GitHub issue with the Airbyte team. [normalization issue] If you have specific evidence that Airbyte 0.32 handled CJK column names differently, I'd suggest checking the Airbyte changelog or opening a discussion on the Airbyte community forum for confirmation from the team. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.