@kapa.ai Airbyte 2.0.1 — Mongo discover hangs, sidecar 9-min file-timeout not raisable via Helm Deployment - Airbyte version: 2.0.1 (Community edition, OSS Helm chart) - Platform: Kubernetes (OVH Managed Kubernetes), single namespace data-applications - Storage backend: built-in MinIO (1Gi PVC) - Database: external PostgreSQL 17.9 (OVH managed) - Components running: airbyte-server, airbyte-worker (2 replicas), airbyte-workload-api-server, airbyte-workload-launcher (1 replica), airbyte-temporal, airbyte-cron, airbyte-connector-builder-server, airbyte-minio-0 - Connector: airbyte/source-mongodb-v2:2.0.7 (cluster_type SELF_MANAGED_REPLICA_SET, schema_enforced=false, discover_sample_size=100, databases=["elax"]) - Sidecar image: airbyte/connector-sidecar:2.0.1 - Init image: airbyte/workload-init-container:2.0.1 Problem Every scheduled sync triggers a fresh schema discover (cron 0 30 */2 * * ?). For one Mongo source the discover pod always fails after exactly 10 minutes with exitCode 2 (main) and 143 (sidecar SIGTERM). Replication itself still succeeds because Airbyte reuses the previously-cached catalog — but that cache is now 8 days stale, so newly-created Mongo collections never reach the catalog and never sync to BigQuery. Sidecar log (smoking gun) 2026-05-12 124141 WARN i.a.c.ConnectorWatcher(waitForConnectorOutput):107 - Failed to find output files from connector within timeout of 9 minute(s). Is the connector still running? 2026-05-12 124141 INFO i.a.c.ConnectorWatcher(failWorkload):342 - Failing workload b2e713cd-..._a2b5a363-..._0_discover. 2026-05-12 124141 INFO i.a.c.ConnectorWatcher(exitFileNotFound):260 - Deliberately exiting process with code 2. Mongo connector behavior - Mongo cluster has 140 collections in the target DB. - source-mongodb-v2 opens client, enumerates 140 collections, then logs once per collection: INFO ForkJoinPool.commonPool-worker-1 i.a.i.s.m.MongoUtil(getFieldsForSchemaless):379 Stream discover timeout value (seconds): 600 - After all 140 init lines (around T+4 min), connector enters silent schemaless sampling phase (no further stdout/stderr). - The connector is still alive and progressing (no errors, mongo driver healthy, primary discovered), but the sidecar's 9-minute "no exit file yet" watchdog fires and kills it before sampling completes. Resource limits on the discover pod main (source-mongodb-v2): requests: cpu=100m, memory=256Mi limits: cpu=500m, memory=1Gi connector-sidecar: requests: cpu=100m, memory=256Mi limits: cpu=250m, memory=512Mi With cpu.limit=500m on the main container, Runtime.availableProcessors() reports 1, so ForkJoinPool.commonPool runs a single worker, sampling collections serially → 140 × per-collection-sample easily exceeds 9 minutes. What we tried / verified 1. ✅ Pod is not OOMKilled. Memory usage well under 1Gi. 2. ✅ Mongo connectivity fine — TLS, replica set primary discovered, no auth errors. 3. ✅ No NetworkPolicy in the namespace, so the recurring getAllowedHosts:70 - All allowedHosts values are un-replaced - [${connection_string}] log (caused by database_config.connection_string being nested under database_config while the connector's allowedHosts template references ${connection_string} at the top level) is harmless here. Worth flagging upstream though. 4. ✅ Catalog last persisted at updatedAt: 1778200473 (2026-05-04 UTC, 8 days ago). Every 2-hour discover since has hit the same 9-minute kill. 5. ✅ Discover failures don't surface in GET /api/v1/jobs/list?configTypes=[discover_schema] (returns 0 jobs). They only show up in the airbyte-server log + as workload pods that get killed. What I want to do Raise the sidecar file-timeout from 9 → 30 minutes for discover (the airbyte.sidecar.file-timeout-minutes property → SIDECAR_FILE_TIMEOUT_MINUTES env var read by ConnectorWatcher). Why the obvious path is blocked - Source airbyte-connector-sidecar/src/main/resources/application.yml: file-timeout-minutes: ${SIDECAR_FILE_TIMEOUT_MINUTES:9} - However, env vars on connector-sidecar pods are constructed by airbyte-workload-launcher from a fixed whitelist. The whitelist enum (airbyte-commons/src/main/kotlin/io/airbyte/commons/envvar/EnvVar.kt) contains SIDECAR_KUBE_CPU_LIMIT and SIDECAR_MEMORY_REQUEST but does NOT contain SIDECAR_FILE_TIMEOUT_MINUTES. Inspecting the live discover pod confirms it: the sidecar container only has 35 env vars, none related to file timeout. - Helm chart airbyte/airbyte 2.0.1 only exposes workloadLauncher.connectorSidecar.image — no extraEnv / env_vars / extraJavaOpts knob for the spawned sidecar container. - Therefore there is no chart-level or env-level workaround in 2.0.1 to raise the timeout. The only options are (a) fork the platform and add SIDECAR_FILE_TIMEOUT_MINUTES to EnvVar + the launcher's sidecar env factory, (b) deploy a mutating admission webhook to inject the env into connector-sidecar containers, or (c) build a custom sidecar image with a different default. Questions for you (Airbyte AI) 1. Is there an undocumented mechanism in Airbyte 2.0.1 to override SIDECAR_FILE_TIMEOUT_MINUTES for discover pods? (e.g. a feature flag in LaunchDarkly, a workspace-level config, an env on workload-launcher that I missed.) 2. Is there a published roadmap to make airbyte.sidecar.file-timeout-minutes configurable via the chart (per-operation ideally, since the in-sync timeout is separately tunable via SIDECAR_FILE_TIMEOUT_WITHIN_SYNC_MINUTES)? The application.yml even has a TODO comment: "Can we bump this value? Does it need to be configured per operation? Should we pass it in from the launcher?" 3. Are there known patterns (e.g. connector-resource-overrides ConfigMap, feature-flag store) that let me raise discover CPU / memory per source, not globally for all discover pods? My only short-term mitigation is raising the global global.workloads.discover.cpu.limit from 500m to 2, which over-allocates for connectors that don't need it. 4. Is the recurring ConfigReplacer.getAllowedHosts - All allowedHosts values are un-replaced - [${connection_string}] log a known bug for source-mongodb-v2 v2.0.7 (where the template references ${connection_string} but the field is nested under database_config.connection_string in newer config shapes)?

This is a detailed and well-researched issue. Let me address each question based on what the knowledge sources actually contain. 1. Undocumented mechanism to override SIDECAR_FILE_TIMEOUT_MINUTES in 2.0.1 The knowledge sources do not document any undocumented mechanism (feature flag, workspace-level config, or workload-launcher env passthrough) to override

SIDECAR_FILE_TIMEOUT_MINUTES

in 2.0.1. Your analysis aligns with what was reported in a similar GitHub issue (filed against Helm chart v1.2.0), where the conclusion was the same: the variable is not exposed via the chart, and the only documented workaround at the time was building a custom sidecar image: [Allow Sidecar Timeout Config]

FROM airbyte/connector-sidecar:2.0.1
ENV SIDECAR_FILE_TIMEOUT_MINUTES=30
ENV SIDECAR_FILE_TIMEOUT_WITHIN_SYNC_MINUTES=60

That issue was subsequently closed as stale without a fix being merged, so option (c) — custom sidecar image — remains the most practical short-term workaround the knowledge sources support. 2. Published roadmap for chart-configurable SIDECAR_FILE_TIMEOUT_MINUTES The knowledge sources do not contain any roadmap information for making this configurable via the Helm chart. The GitHub issue requesting this feature was closed as stale after 20 days of inactivity [Allow Sidecar Timeout Config], which suggests it has not been prioritized. There is no evidence in the available sources of a per-operation timeout knob being added in chart v2. 3. Per-source CPU/memory overrides for discover pods The knowledge sources do not document a per-source connector resource override mechanism. The Helm chart v2 does expose global discover resource settings (e.g.,

global.workloads.resources.discover.cpu.limit

), but these apply globally to all discover pods, not per-source. [Update values.yaml] There is an open GitHub issue reporting that even these global resource settings may not be correctly propagated to pods in abctl/v2 deployments [Resource Limits Ignored], though that issue is specific to

abctl

rather than a direct Helm install. Your proposed mitigation of raising

global.workloads.discover.cpu.limit

globally is the only chart-level option the knowledge sources support. 4. getAllowedHosts / ${connection_string} log for source-mongodb-v2 v2.0.7 The knowledge sources do not contain any information about this specific log warning or whether it is a known bug in source-mongodb-v2 v2.0.7. I cannot confirm or deny this from the available evidence. Summary of actionable options supported by the knowledge sources: 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements