Hello @kapa.ai, I hope you are well. Anyone know how to increase Nginx timeout to avoid 504 error when streams are discovered ( ex: S3, SFTP Bulk, ...). I'm using an abctl deployment Thanks in advance for your reply

Hello, To increase the Nginx timeout and avoid the 504 error during stream discovery, you can modify the timeout settings in the Nginx configuration. For an

abctl

deployment, you can increase the timeout by editing the ingress definition. Here's how you can do it:

shell kubectl -n airbyte-abctl edit <http://ingresses.networking.k8s.io|ingresses.networking.k8s.io> ingress-abctl

Then, add the following annotations:

yaml apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1> kind: Ingress metadata:   annotations:     <http://nginx.ingress.kubernetes.io/proxy-read-timeout|nginx.ingress.kubernetes.io/proxy-read-timeout>: "120"

This will set the proxy read timeout to 120 seconds. You can adjust this value according to your needs. Additionally, you can also set the

HTTP_IDLE_TIMEOUT

environment variable in your helm chart values:

yaml global:   env_vars:     HTTP_IDLE_TIMEOUT: 1800s

And set the

proxy-send-timeout

and

proxy-read-timeout

in the webapp ingress:

yaml webapp:   ingress:     annotations:       <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: internal       <http://nginx.ingress.kubernetes.io/proxy-body-size|nginx.ingress.kubernetes.io/proxy-body-size>: 16m       <http://nginx.ingress.kubernetes.io/proxy-send-timeout|nginx.ingress.kubernetes.io/proxy-send-timeout>: 1800       <http://nginx.ingress.kubernetes.io/proxy-read-timeout|nginx.ingress.kubernetes.io/proxy-read-timeout>: 1800

Please note that these values are just examples and you might need to adjust them according to your specific needs. For more information, you can refer to these discussions and issues on GitHub: [Discussion 40738](https://github.com/airbytehq/airbyte/discussions/40738), [Issue 45156](https://github.com/airbytehq/airbyte/issues/45156), [Issue 29712](https://github.com/airbytehq/airbyte/issues/29712). I hope this helps! Let me know if you have any other questions.