Is there an option to set timeout mechanism for Syncs?
We are self-hosting Airbyte on K8s, and recently encountered with a sync that has been running for last 3 days. But when I checked the k8s pods, there is no associated pod running - I guess somehow got killed. And, the logs in the sync also had paused and it was as if some sync was going on.
I manually cancelled the sync so it reran again and finally completed in just a few minutes. But it's concerning that a a problematic sync job remains in running state for a long time without alarming us.
Augustin Lafanechere (Airbyte)
03/21/2022, 3:00 PM
Hi @Jay Bujala, by default Airbyte sync will timeout after 3 days. You can tune this according to your need with the
environment variable mapped to the worker and scheduler pods.
03/21/2022, 5:01 PM
Thanks @Augustin Lafanechere (Airbyte) for the prompt response. One of our teammates apparently pushed a rolling update to all nodes so that caused this issue.