Hi team, how are you? I'm exploring airbyte with a...
# ask-community-for-troubleshooting
m
Hi team, how are you? I'm exploring airbyte with another ETL tools, and I got a doubt. When I try to get data from a source and put it into an S3 bucket it doesn't show the field to use dbt transform scripts? There is a reason why ? I meam if I choose to do some transformation like a traditional ETL instead of an ELT it won't be possible with S3 destination? My case is that I need to anonymize some data before get it into my Lake.
1
u
Hi @Marcos Soares de Oliveira Junior, DBT transformations are powered by SQL and can only work on query engines (such as postgres, bigquery etc.), this is why we do not offer transformation for cloud storage destinations.
This is a limitation we're aware of. You can track the following issues to receive updates on this topic: • https://github.com/airbytehq/airbyte/issues/2566https://github.com/airbytehq/airbyte/issues/2645 If you are an Airflow user you can use our
AirbyteTriggerSyncOperator
to launch sync operation from Airflow and perform post replication transformations with whatever tool you want.
1
m
Thanks @[DEPRECATED] Augustin Lafanechere I'll track this issues!