@kapa.ai
When trying to trigger connections from airflow, I'm getting below error. It was working fine till morning, started getting errors all of a sudden.
default-hostname
* Reading remote logs from Cloud Logging.
[2026-04-30, 142117 UTC] {local_task_job_runner.py:123} ▶️ Pre task execution logs
[2026-04-30, 142118 UTC] {base.py:84} INFO - Retrieving connection 'airbyte_default'
[2026-04-30, 142333 UTC] {http.py:356} WARNING - HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')) Tenacity will retry to execute the operation
[2026-04-30, 142333 UTC] {taskinstance.py:3315} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 493, in _make_request
conn.request(
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 494, in request
self.endheaders()
File "/opt/python3.11/lib/python3.11/http/client.py", line 1293, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/python3.11/lib/python3.11/http/client.py", line 1052, in _send_output
self.send(msg)
File "/opt/python3.11/lib/python3.11/http/client.py", line 990, in send
self.connect()
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 325, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 207, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 644, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 769, in _execute_task
*result = _execute_callable(context=context, *execute_callable_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 735, in _execute_callable
return ExecutionCallableRunner(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/utils/operator_helpers.py", line 252, in run
*return self.func(*args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 424, in wrapper
*return func(self, *args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/airbyte/operators/airbyte.py", line 71, in execute
job_object = hook.submit_sync_connection(connection_id=self.connection_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 92, in submit_sync_connection
return self.run(
^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 301, in run
return self.run_and_check(session, prepped_request, self.merged_extra)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 357, in run_and_check
raise ex
File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 349, in run_and_check
*response = session.send(prepped_request, *send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
*r = adapter.send(request, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 665, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)'))
[2026-04-30, 142333 UTC] {taskinstance.py:1227} INFO - Marking task as UP_FOR_RETRY. dag_id=ll_rpc_pc_gcs_es_ts_pipeline_v2, task_id=trigger_airbyte_product_catalog_mysql_to_gcs_sync, run_id=scheduled__2026-04-30T133000+00:00, execution_date=20260430T133000, start_date=20260430T142117, end_date=20260430T142333
[2026-04-30, 142333 UTC] {taskinstance.py:341} ▼ Post task execution logs
[2026-04-30, 142333 UTC] {standard_task_runner.py:124} ERROR - Failed to execute job 355511 for task trigger_airbyte_product_catalog_mysql_to_gcs_sync (HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')); 44870)
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 493, in _make_request
conn.request(
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 494, in request
self.endheaders()
File "/opt/python3.11/lib/python3.11/http/client.py", line 1293, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/python3.11/lib/python3.11/http/client.py", line 1052, in _send_output
self.send(msg)
File "/opt/python3.11/lib/python3.11/http/client.py", line 990, in send
self.connect()
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 325, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connection.py", line 207, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/requests/adapters.py", line 644, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='35.200.160.216', port=80): Max retries exceeded with url: /api/v1/connections/sync (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7cf8aab510>, 'Connection to 35.200.160.216 timed out. (connect timeout=None)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python3.11/lib/python3.11/site-packages/airflow/task/task_runner/standard_task_runner.py", line 117, in _start_by_fork
ret = args.func(args, dag=self.dag)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/cli_config.py", line 49, in command
*return func(*args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/utils/cli.py", line 116, in wrapper
*return f(*args, *kwargs)
^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 489, in task_run
task_return_code = _run_task_by_selected_method(args, _dag, ti)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python3.11/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 256, in _run_task_by_selected_method
return _run_raw_task(args, ti)