https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
t

Thien Pham

11/01/2021, 8:29 AM
Hello team, I just hear about Airbyte some days ago, but cannot find anything describe the relationship between Airflow & Airbyte. I'm imaging that we should create an Airflow DAG to enable an Airbyte Job, and this job will run by Airbyte Scheduler & Worker. However if we do something like that, we will have Airflow Webserver, Scheduler, Worker & Airbyte Webserver, Scheduler, Worker. Do we have redundancy Scheduler & Worker? Or doing something like that will have some pros that I didn't notice?
👀 1
I hope this can help
t

Thien Pham

11/04/2021, 2:30 AM
thank you, I will try it and feedback with you if it will help or not ❤️
Please correct me if I'm wrong, above document is exactly what I'm imaging about the relationship between Airflow & Airbyte, but what i'm mention here is about the redundancy Scheduler & Worker if we do something like that. Do you have any idea about that, maybe it have some pros (& cons)?
u

[DEPRECATED] Marcos Marx

11/04/2021, 8:41 PM
Thien, Airflow is a different project so we cannot use their scheduler/worker. If you want to have high-availability of Airbyte you can deploy using Kubernetes.
👍 1
3 Views