Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

image.png

I try to use airbyte to synchronize data from shopify to postgres, the basic raw_data has no problem, but in the use of transformation of normalized tabular data, will generate a lot of price_set and other sub-tables, I want to avoid the transformation of these json objects, how should I operate it, have you encountered the same problem partners?

Hello <@U04CLCF95J6>, Airbyte will try to generate all nested tables from parent tables. You can export the normalization module (dbt) Airbyte generates and edit it to remote all those tables and use Custom Transformation to achieve your goal. Check more about here: <https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-dbt>

1. Your suggestion I also think is a good option, but customizing or modifying the dbt I think is an operation beyond my capabilities.
2. I processed the scheam file in the source connector, reconstructed a new connector by changing the type of the fields that don't need to be expanded, from object to string, and removing the corresponding expanded attributes. I'm not sure if this makes sense or if it creates invisible problems, but it does meet my needs.

<@U02TQLBLDU4>
1. Your suggestion I also think is a good option, but customizing or modifying the dbt I think is an operation beyond my capabilities.
2. I processed the scheam file in the source connector, reconstructed a new connector by changing the type of the fields that don't need to be expanded, from object to string, and removing the corresponding expanded attributes. I'm not sure if this makes sense or if it creates invisible problems, but it does meet my needs.
<https://airbytehq.slack.com/files/U04CLCF95J6/F04DQQRSH5E/image.png>