<@U04S8PLJWMT> 5 connectors failed to sync with er...
# ask-ai
d
@kapa.ai 5 connectors failed to sync with error Warning from source: Check took too long. Check exceeded the timeout.
k
“Check took too long” means the connector’s connection check (or schema refresh) didn’t finish before Airbyte’s timeout window expired. This is a platform-level timeout, not necessarily a bug in all 5 connectors. From the docs and issues you shared, there are three main timeout mechanisms relevant here: 1. HTTP / Schema discovery timeouts (backend / server) If a source has many tables/columns or a slow API, the one-time
discover
or check call can exceed the default HTTP timeouts. You can increase these in your Helm
values.yaml
(self‑hosted / enterprise): - For Helm chart v1 (examples for Airbyte 1.6–Next):
Copy code
yaml      server:        extraEnv:          - name: HTTP_IDLE_TIMEOUT            value: 20m          - name: READ_TIMEOUT            value: 30m
[[Scaling Airbyte](https://docs.airbyte.com/platform/1.7/enterprise-setup/scaling-airbyte#schema-discovery-timeouts); [Next scaling](https://docs.airbyte.com/platform/next/enterprise-setup/scaling-airbyte#schema-discovery-timeouts)] - For Helm chart v2 (1.8+/Next):
Copy code
yaml      server:        httpIdleTimeout: 20m        extraEnv:          - name: READ_TIMEOUT            value: 30m
[[1.8 scaling](https://docs.airbyte.com/platform/1.8/enterprise-setup/scaling-airbyte#schema-discovery-timeouts); [Enterprise scaling](https://docs.airbyte.com/platform/enterprise-setup/scaling-airbyte#schema-discovery-timeouts)] This specifically helps with long‑running discover / schema refresh / check operations. 2. Heartbeats and long‑running syncs During syncs (not checks), the platform monitors activity from connectors: - Source heartbeat: if no
RECORD
or
STATE
messages are seen for longer than
heartbeat-max-seconds-between-messages
, the sync is failed with “The source is unresponsive”. You can tune this in `flags.yaml`:
Copy code
yaml      heartbeat-max-seconds-between-messages: <seconds>      heartbeat.failSync: true|false
[[Heartbeats](https://docs.airbyte.com/platform/understanding-airbyte/heartbeats#technical-details)] - Destination timeout: if a call to the destination takes longer than
destination-timeout-max-seconds
, the sync can fail with “The destination is unresponsive”. Also in `flags.yaml`:
Copy code
yaml      destination-timeout-max-seconds: <seconds>      destination-timeout.failSync: true|false
[[Heartbeats](https://docs.airbyte.com/platform/understanding-airbyte/heartbeats#technical-details)] These don’t directly control the “check took too long” UI message, but they show how platform-level timeouts behave and can be tuned. 3. Per‑connector internal timeouts Some connectors have their own hard‑coded or decorated timeouts (e.g., Google Ads
request_records_job
timing out after 5 minutes in large datasets). [[Google Ads issue](https://github.com/airbytehq/airbyte/issues/49254)] These are connector‑specific and can’t usually be changed from the UI; you’d need a connector update. --- ### How to proceed for your case (“Check took too long” for 5 connectors) Since multiple connectors are failing the check with the same timeout wording, it’s likely a platform/server timeout rather than 5 independent connector bugs: 1. If you’re self‑hosting with Helm: - Increase
HTTP_IDLE_TIMEOUT
and
READ_TIMEOUT
as above and redeploy. [[Scaling Airbyte](https://docs.airbyte.com/platform/next/enterprise-setup/scaling-airbyte#schema-discovery-timeouts)] - Then re‑run: - “Check connection” - And, if relevant, “Refresh schema” 2. If you’re on a managed/cloud deployment: The shared docs don’t describe changing these limits from the UI or API; I’m sorry, but there’s no information in the provided sources about tuning timeouts in Airbyte Cloud. In that case, you’d typically need to open a support ticket with your environment details and example connection IDs. 3. Check connector/API specifics: - For APIs that run long asynchronous jobs, consider using the Connector Builder async streams “Polling” tab so the connector polls job status instead of a single, long‑running request. [[Async polling](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/async-streams#polling-tab)] - Ensure the API you’re calling isn’t itself returning 504 / Gateway Timeout, as seen in the Facebook Marketing example. [[Facebook issue](https://github.com/airbytehq/airbyte/issues/53132)] --- If you share whether you’re using self‑hosted (and how you deploy: Docker, Helm, etc.) and which Airbyte version, I can narrow down exactly which
values.yaml
/
flags.yaml
settings apply.
d
self hosted using kubernets and using version 1.2.0 It looks like every connector is not timing out on the platform @kapa.ai
k
A message from kapa.ai
Given you’re self‑hosting on Kubernetes with Airbyte 1.2.0 and all connectors’ Checks are timing out, this strongly points to platform‑level timeouts rather than individual connectors. For 1.2.0 + Helm, you can address long‑running check/discover calls by increasing the server HTTP timeouts in
values.yaml
as documented for schema discovery timeouts (the mechanism is the same for long checks): ### 1. Increase server HTTP timeouts (Helm) For Helm chart v1 (which applies around 1.2.0), add or update:
Copy code
yaml server:   extraEnv:     - name: HTTP_IDLE_TIMEOUT       value: 20m     - name: READ_TIMEOUT       value: 30m
This is the documented way to let long‑running
discover
/ check operations complete without hitting the default server timeout. [[Scaling Airbyte](https://docs.airbyte.com/platform/enterprise-setup/scaling-airbyte#schema-discovery-timeouts)] > Note: Newer docs show the same pattern for 1.6–Next, so this is the recommended approach for 1.2.0 as well. [[1.6 scaling](https://docs.airbyte.com/platform/1.6/enterprise-setup/scaling-airbyte#schema-discovery-timeouts); [Next scaling](https://docs.airbyte.com/platform/next/enterprise-setup/scaling-airbyte#schema-discovery-timeouts)] After editing
values.yaml
, redeploy your Helm release so the
airbyte-server
pod picks up the new env vars. ### 2. Be aware of the connector sidecar timeout on 1.2.0 On 1.2.0, there is also a connector sidecar timeout that affects large schema discovery/checks (e.g., BigQuery, Netsuite): -
SIDECAR_FILE_TIMEOUT_MINUTES
(defaults to ~9 minutes) -
SIDECAR_FILE_TIMEOUT_WITHIN_SYNC_MINUTES
These are not exposed via Helm 1.2.0 and have been reported to cause timeouts in large projects. [[Sidecar timeout issue](https://github.com/airbytehq/airbyte/issues/48855); [Netsuite timeout](https://github.com/airbytehq/airbyte/issues/55191)] The documented workaround in that GitHub issue is to build a custom sidecar image with larger env values:
Copy code
dockerfile FROM airbyte/connector-sidecar:1.2.0  ENV SIDECAR_FILE_TIMEOUT_MINUTES=30 ENV SIDECAR_FILE_TIMEOUT_WITHIN_SYNC_MINUTES=60
Then configure your deployment to use that image for the connector sidecar. The exact Helm knobs for swapping that image are not described in the provided sources, so I can’t give a precise values.yaml snippet; only the general approach is documented. ### 3. Check worker concurrency / resource pressure If checks are backing up or never completing because workers can’t start, ensure your worker concurrency and resources are reasonable:
Copy code
yaml worker:   extraEnv:     - name: MAX_CHECK_WORKERS       value: "5"     - name: MAX_SYNC_WORKERS       value: "5"   replicaCount: 2
This pattern is documented for scaling concurrent sync/check workers and avoiding stuck jobs due to under‑provisioning. [[Scaling concurrent syncs](https://docs.airbyte.com/platform/enterprise-setup/scaling-airbyte#concurrent-sync-limits)] Also verify your
jobs.resources
(CPU/memory) limits/requests are not so high that pods can’t be scheduled. [[Scaling Airbyte](https://docs.airbyte.com/platform/enterprise-setup/scaling-airbyte#connector-cpu--memory-settings)] --- Summary of concrete steps for 1.2.0 on Kubernetes: 1. Increase
HTTP_IDLE_TIMEOUT
and
READ_TIMEOUT
in
server.extraEnv
and redeploy. 2. If large schemas (BigQuery, Netsuite, etc.) are involved, consider the documented sidecar workaround to raise
SIDECAR_FILE_TIMEOUT_MINUTES
. 3. Ensure
MAX_CHECK_WORKERS
and worker resources are set to values your cluster can actually run. If you can share how you’re deploying Airbyte 1.2.0 on K8s (Helm vs. raw manifests) and whether you use the official Helm chart v1.2.0, I can help you translate this into a more specific
values.yaml
example based on the docs above.