https://linen.dev logo
Join Slack
Powered by
Hi! :wave: It seems that normalized tables get th...
# ask-community-for-troubleshooting
f
Hi! 👋 It seems that normalized tables get their names truncated to 43 characters when they have a prefix. Is that an expected behavior? (More details in thread ⬇️ )
I am creating a connection between a Postgres instance and Clickhouse. The postgres table name is 
municipality_zipcodes
, the prefix is 
bf476d90_13b7_4e2e_b4ca_eeeeed0cffb1___
. If I look into the UI, there is no problem, the stream name is 
bf476d90_13b7_4e2e_b4ca_eeeeed0cffb1___municipality_zipcodes
. Nevertheless, when I look in Clickhouse, the prefix gets truncated and I end up with 
bf476d90_13b7_4e2e_b__municipality_zipcodes
. What's strange is that the 
_airbyte_raw
table is good though: I do get a 
_airbyte_raw_bf476d90_13b7_4e2e_b4ca_eeeeed0cffb1___municipality_zipcodes
table. I assume the problem comes from the transformation part? Is that a bug or a feature?
All transformed tables get truncated to 43 characters, and it's the prefix that gets truncated. All 
_airbyte_raw
tables have no issues with their names.
Examples from clickhouse
Sometimes both the prefix AND the original stream name get truncated
c
Yes this is due to a limit here: https://github.com/airbytehq/airbyte/blob/74c419572165dee3b668fcb86fd9eb28a0a8d39b[…]normalization/transform_catalog/destination_name_transformer.py i’m not sure why clickhouse’s limit is set to be the same as Postgres… from what you are reporting it seems like we could set a higher limit before the truncating operations kick in you can open a pull request / issue to tweak that
f
Thank you for this insanely fast response!
Actually it seems that Clickhouse has no limit on identifiers. Should we set it to 255 to give some more room?
c
yes, if you can submit a pull request with a higher limit, the team will assist you and run the integration tests to verify that won’t break anything
5 Views
Open in Slack
PreviousNext
Powered by