Another question, where are normalization and cust...
# ask-community-for-troubleshooting
l
Another question, where are normalization and custom transformations done? source or destination? Because my destination is Redshift, and it´s too pricy
u
Hello again! Normalization is done in the destination connector.
l
Hi.. and also the custom transformation is done at the destination? I just can´t figure out how to reduce resources. My destination is Redshift and it´s too pricy
u
Yes, transformation is also done at the destination. I do not have any docs or best practices to share on how to cut costs, but we do have some great docs on normalization and transformation. Here's a chart: https://docs.airbyte.com/understanding-airbyte/basic-normalization/#why-does-airbyte-have-basic-normalization Here's a three part doc: https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql/ And here's a great normalization overview: https://docs.airbyte.com/understanding-airbyte/basic-normalization
l
Hi, I´ll try to explain my question: Let´s say I have 4 tables in google ads source, and redshift destination (with S3 Staging upload method). I set the connection (incremental + history deduped). I also have basic normalization and I set in the connection a dbt transformation that creates a table with fields from the 4 source tables. In redshift I only need the table I created on dbt, which might be a table of 20 rows. Are the other 4 tables that I synced from source going to be written in Redshift destination?
u
Yes - they are first written as json blobs, then normalized, all within the destination. Maybe you've notice tables named
airbyte_raw
or something similar? This is an ETL best practice - to have raw data available, so that's why it functions this way. Does this answer your question?
l
yes, it answered. Thank you
Is there a workaround to this? because I only need the view, not the raw tables + scd + stg + normalization of the 4 source tables..
I mean, is there a way to filter the data before writing it on the destination?
u
No, there is not unfortunately. The reason is that writing raw data to the destination is an ETL best practice that Airbyte must follow.