<@U04S8PLJWMT> When trying to trigger connections ...
# replication-ask-ai
v
@kapa.ai When trying to trigger connections from airflow, I'm getting below error. It was working fine till morning, started getting errors all of a sudden. default-hostname * Reading remote logs from Cloud Logging. [2026-04-30, 142117 UTC] {local_task_job_runner.py:123} ▶️ Pre task execution logs [2026-04-30, 142118 UTC] {base.py:84} INFO - Retrieving connection 'airbyte_default' [2026-04-30, 142333 UTC] {http.py:356} WARNING - HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')) Tenacity will retry to execute the operation [2026-04-30, 142333 UTC] {taskinstance.py:3315} ERROR - Task failed with exception Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 493, in _make_request conn.request( File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 494, in request self.endheaders() File "/opt/python3.11/lib/python3.11/http/client.py", line 1293, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/opt/python3.11/lib/python3.11/http/client.py", line 1052, in _send_output self.send(msg) File "/opt/python3.11/lib/python3.11/http/client.py", line 990, in send self.connect() File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 325, in connect self.sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 207, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)') The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 769, in _execute_task *result = _execute_callable(context=context, *execute_callable_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 735, in _execute_callable return ExecutionCallableRunner( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/utils/operator_helpers.py", line 252, in run *return self.func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 424, in wrapper *return func(self, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/airbyte/operators/airbyte.py", line 71, in execute job_object = hook.submit_sync_connection(connection_id=self.connection_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 92, in submit_sync_connection return self.run( ^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 301, in run return self.run_and_check(session, prepped_request, self.merged_extra) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 357, in run_and_check raise ex File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 349, in run_and_check *response = session.send(prepped_request, *send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/requests/sessions.py", line 703, in send *r = adapter.send(request, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 665, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')) [2026-04-30, 142333 UTC] {taskinstance.py:1227} INFO - Marking task as UP_FOR_RETRY. dag_id=ll_rpc_pc_gcs_es_ts_pipeline_v2, task_id=trigger_airbyte_product_catalog_mysql_to_gcs_sync, run_id=scheduled__2026-04-30T133000+00:00, execution_date=20260430T133000, start_date=20260430T142117, end_date=20260430T142333 [2026-04-30, 142333 UTC] {taskinstance.py:341} ▼ Post task execution logs [2026-04-30, 142333 UTC] {standard_task_runner.py:124} ERROR - Failed to execute job 355511 for task trigger_airbyte_product_catalog_mysql_to_gcs_sync (HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')); 44870) Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 493, in _make_request conn.request( File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 494, in request self.endheaders() File "/opt/python3.11/lib/python3.11/http/client.py", line 1293, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/opt/python3.11/lib/python3.11/http/client.py", line 1052, in _send_output self.send(msg) File "/opt/python3.11/lib/python3.11/http/client.py", line 990, in send self.connect() File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 325, in connect self.sock = self._new_conn() ^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 207, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)') The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/airflow/task/task_runner/standard_task_runner.py", line 117, in _start_by_fork ret = args.func(args, dag=self.dag) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/cli_config.py", line 49, in command *return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/utils/cli.py", line 116, in wrapper *return f(*args, *kwargs) ^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 489, in task_run task_return_code = _run_task_by_selected_method(args, _dag, ti) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 256, in _run_task_by_selected_method return _run_raw_task(args, ti)
k
The error you're seeing is a connection timeout — Airflow cannot reach your Airbyte instance at
35.200.160.216:80
. The error occurs at the TCP connection level (before any HTTP request is even sent), which means this is a network/infrastructure issue, not an Airbyte configuration problem. Here are the most likely causes and things to check: 1. Verify Airbyte is running Check if your Airbyte instance is up and healthy. If it restarted or crashed, the IP may no longer be serving traffic on port 80. 2. Check network connectivity From your Airflow environment, try to reach the Airbyte host:
Copy code
curl http://35.200.160.216/api/v1/health
If this times out too, the issue is network-level (firewall rules, routing, VM stopped, etc.). 3. Check firewall / security group rules If Airbyte is hosted on a cloud VM (e.g., GCP, given the IP range), verify that: • The VM is still running. • Firewall/security group rules still allow inbound traffic on port 80 from your Airflow host. • No VPC or network policy changes were made this morning. 4. Verify the Airflow connection configuration In the Airflow UI, go to Admin > Connections and confirm the
airbyte_default
connection still points to the correct host and port. [Airflow connection setup] 5. Consider upgrading to the newer provider The error path (
/api/v1/connections/sync
) indicates you're using an older version of the Airbyte Airflow provider that talks to the internal Config API. The newer
apache-airflow-providers-airbyte>=5.0.0
uses the public Airbyte API and supports both Cloud and Self-Managed deployments with Client ID/Secret authentication. [Airflow Airbyte Provider] Since it was working fine earlier today and stopped suddenly, the most probable cause is an infrastructure change (VM stopped, IP changed, firewall rule modified) rather than a code or configuration issue on the Airflow side. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.