Is there a way to have a max runtime for a connection? I'm having an issue where sometimes the API will become unresponsive and continue spinning without a timeout. I need to implement logic to tell it to fail if it takes too long

Yes, Airbyte provides mechanisms to enforce a maximum runtime for connections and to fail syncs that take too long. 1. Heartbeat and Timeout Configuration (flags.yaml): - Airbyte uses a heartbeat mechanism for both sources and destinations. If the source does not emit a

RECORD

or

STATE

message within a configured threshold (default: 3 hours), the sync will fail with a message like "The source is unresponsive." - For destinations, if a call takes longer than a configured timeout (default: 24 hours), the sync will fail with "The destination is unresponsive." - You can configure these thresholds in the

flags.yaml

file:

yaml   heartbeat-max-seconds-between-messages: <seconds>  # e.g., 3600 for 1 hour   destination-timeout-max-seconds: <seconds>         # e.g., 7200 for 2 hours

Setting these to lower values will cause Airbyte to fail syncs that exceed your desired maximum runtime. Both options are enabled to fail syncs by default, but you can control this with

heartbeat.failSync

and

destination-timeout.failSync

options as well. [See documentation for details](https://docs.airbyte.com/platform/understanding-airbyte/heartbeats#technical-details). 2. Global Sync Job Timeout (Environment Variable): - There is also a global environment variable:

SYNC_JOB_MAX_TIMEOUT_DAYS

(minimum value is 1 day), which sets a hard upper limit for sync job duration. This is less granular but can be used to prevent jobs from running indefinitely. [See forum discussion](https://discuss.airbyte.io/t/709). 3. Connector-Specific Timeouts: - Some connectors (e.g., Google Ads) have their own internal timeout settings, which may be hardcoded or configurable. If you are experiencing timeouts with a specific connector, check its documentation or open issues for connector-specific options or workarounds. [See example for Google Ads connector](https://github.com/airbytehq/airbyte/issues/49254). Summary: To implement a maximum runtime and fail unresponsive syncs, adjust the heartbeat and destination timeout values in

flags.yaml

for your Airbyte deployment. This will ensure that syncs are automatically failed if they exceed your specified duration or become unresponsive.

@kapa.ai am i able to put decimal values for this variable? SYNC_JOB_MAX_TIMEOUT_DAYS

A message from kapa.ai