https://linen.dev logo
Join Slack
Title
#advice-data-transformation
Joey Taleño

Joey Taleño

04/13/2022, 5:08 AM
Hello! If Airbyte is for EL, does it mean that Airflow is still the suitable partner for Data Transformation and Orchestration?
😮ctavia-hello: 1
Augustin Lafanechere (Airbyte)

Augustin Lafanechere (Airbyte)

04/13/2022, 2:07 PM
Hi @Joey Taleño, it depends of the complexity of your current data stack. You could consider Airbyte as an orchestrator if you only rely on Airbyte's integration. You could aslo consider as Airbyte as your transformation partner if you rely on our custom transformation with DBT. But you can of course use Airflow for more global orchestration and other transformation tools if you want to have post-load downstream pipelines. We have an Airflow operator in case you did not know.
👍 2
j

Jordan Fox

04/13/2022, 4:43 PM
I tend to think of Airbyte as a first class data ingestion tool and a second class anything else tool. It's prime directive is to ingest data from source to destination. If you have use cases for orchestration and use cases for data transformation, look into other tools like dagster, prefect, airflow (orchestration) and dbt (transformation).
👍 3
View count: 1
Join thread in Slack