https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> I am getting this error during sync...
# replication-ask-ai
s
@kapa.ai I am getting this error during sync 2026-03-13 125151 replication-orchestrator INFO Records read: 905000 (15 GB) 2026-03-13 125201 replication-orchestrator ERROR Source process exited with non-zero exit code 143 2026-03-13 125201 replication-orchestrator ERROR SourceReader error: 2026-03-13 125201 replication-orchestrator INFO SourceReader finished.
k
The error 
Source process exited with non-zero exit code 143
is a known issue in Airbyte. Exit code 143 typically indicates the process was killed, most commonly due to Out of Memory (OOM) conditions or a job cancellation/timeout. [Sync Jobs Failing] Given that your log shows 905,000 records read at 15 GB, memory pressure is the most likely culprit. Common Causes & Remediation Steps 1. Out of Memory (OOM) Processing 15 GB of data can exhaust the memory allocated to the sync job container. ā€¢ If self-hosting (OSS/Kubernetes): Increase the memory limits for your sync job containers by raising the 
JOB_MAIN_CONTAINER_MEMORY_REQUEST
and 
JOB_MAIN_CONTAINER_MEMORY_LIMIT
environment variables (e.g., try 8 GB or higher). [Sync Jobs Failing; Source exit code 143] 2. Job Timeout / Cancellation Exit code 143 can also occur when a job is cancelled (manually or due to a timeout). The Airbyte logs themselves note: "This warning is normal if the job was cancelled." [Running sync worker cancellation] Check whether a timeout policy or manual cancellation triggered the failure. 3. SourceReader Hanging After Destination Failure There is a known bug where the 
SourceReader
can hang after a destination failure because blocking reads are not interruptible by coroutine cancellation, eventually causing the process to be killed. [SourceReader hang issue] 4. Reduce Data Volume Per Sync If the volume is consistently too large, consider reducing the batch size or date range per sync to process smaller chunks of data at a time. [OOM Exit Code 143] Summary To narrow down the root cause, check the full sync logs for any destination-side errors that may have preceded the source failure, and verify your container memory configuration. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
s
@kapa.ai full error tace
Copy code
INFO Failures: [ {
  "failureOrigin" : "source",
  "failureType" : "system_error",
  "internalMessage" : "Exception while syncing stream companies: Ran out of input",
  "externalMessage" : "Something went wrong in the connector. See the logs for more details.",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 35,
    "from_trace_message" : true,
    "connector_command" : "read"
  },
  "stacktrace" : "airbyte_cdk.sources.concurrent_source.stream_thread_exception.StreamThreadException: Exception while syncing stream companies: Ran out of input\n",
  "timestamp" : 1773356372933,
  "streamDescriptor" : {
    "name" : "companies"
  }
}, {
  "failureOrigin" : "source",
  "internalMessage" : "Source process exited with non-zero exit code 143",
  "externalMessage" : "Something went wrong within the source connector",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 35,
    "connector_command" : "read"
  },
  "stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 143\n\tat io.airbyte.container.orchestrator.worker.SourceReader.run(ReplicationTask.kt:210)\n\tat io.airbyte.container.orchestrator.worker.SourceReader$run$1.invokeSuspend(ReplicationTask.kt)\n\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)\n\tat kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n\tSuppressed: io.airbyte.workers.internal.exception.DestinationException: Destination process exited with non-zero exit code 143\n\t\tat io.airbyte.container.orchestrator.worker.DestinationReader.run(ReplicationTask.kt:55)\n\t\tat io.airbyte.container.orchestrator.worker.ReplicationWorker$runJobs$2$tasks$1$1.invokeSuspend(ReplicationWorker.kt:154)\n\t\tat io.airbyte.container.orchestrator.worker.ReplicationWorker$runJobs$2$tasks$1$1.invoke(ReplicationWorker.kt)\n\t\tat io.airbyte.container.orchestrator.worker.ReplicationWorker$runJobs$2$tasks$1$1.invoke(ReplicationWorker.kt)\n\t\tat io.airbyte.container.orchestrator.worker.util.AsyncUtils$runAsync$1$1.invokeSuspend(AsyncUtils.kt:22)\n\t\tat io.airbyte.container.orchestrator.worker.util.AsyncUtils$runAsync$1$1.invoke(AsyncUtils.kt)\n\t\tat io.airbyte.container.orchestrator.worker.util.AsyncUtils$runAsync$1$1.invoke(AsyncUtils.kt)\n\t\tat kotlinx.coroutines.intrinsics.UndispatchedKt.startUndspatched(Undispatched.kt:66)\n\t\tat kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:43)\n\t\tat kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:165)\n\t\tat kotlinx.coroutines.BuildersKt.withContext(Unknown Source)\n\t\tat io.airbyte.container.orchestrator.worker.util.AsyncUtils$runAsync$1.invokeSuspend(AsyncUtils.kt:21)\n\t\t... 5 more\n",
  "timestamp" : 1773366723344
} ]
k
Looking at your full error trace, there are actually two distinct failures happening: Failure 1: "Ran out of Input" (Root Cause)
Copy code
airbyte_cdk.sources.concurrent_source.stream_thread_exception.StreamThreadException: 
Exception while syncing stream companies: Ran out of input
This is the primary/root cause. The 
Ran out of input
error indicates the source connector encountered an unexpected end of data while reading the 
companies
stream ā€” this is typically a data parsing or connection issue at the source level (e.g., a truncated response, a broken network connection, or a malformed record). Failure 2: Exit Code 143 (Consequence)
Copy code
Source process exited with non-zero exit code 143
Suppressed: Destination process exited with non-zero exit code 143
Both the source and destination processes were killed (exit code 143) as a consequence of the first failure. This is the cascading effect after the source connector crashed. What to Do 1. Investigate the 
companies
stream specifically ā€” the error is isolated to that stream. Check if there are any unusually large or malformed records in your 
companies
data. 2. Check source connector logs in detail ā€” look for more context around the 
Ran out of input
error before the process was killed. 3. Memory may still be a factor ā€” given your earlier log showed 15 GB of data being read, memory pressure could be causing the source to crash mid-read, resulting in the "Ran out of input" error. If self-hosting, consider increasing 
JOB_MAIN_CONTAINER_MEMORY_LIMIT
. [OOM Exit Code 143] 4. Try syncing the 
companies
stream in isolation with a smaller date range or reduced scope to see if the error is reproducible with smaller data volumes. [Sync Jobs Failing] The knowledge sources do not contain specific information about the 
Ran out of input
error, so I'd recommend opening a GitHub issue with Airbyte if the above steps don't resolve it. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai can you change source container restart policy in the job pod
k
The knowledge sources do not explicitly mention a way to change the restart policy specifically for the source container within a job pod. However, what the documentation does show is that Airbyte job pods are created with 
restartPolicy: Never
(as visible in the GitHub issue traces in the knowledge sources). This is a pod-level setting, not a per-container setting in Kubernetes ā€” Kubernetes does not support per-container restart policies. What you can configure for job pods via environment variables (or their Helm chart V2 equivalents) includes things like: ā€¢ Tolerations: 
JOB_KUBE_TOLERATIONS
ā†’ 
global.jobs.kube.tolerations
ā€¢ Node selectors: 
JOB_KUBE_NODE_SELECTORS
ā†’ 
global.jobs.kube.nodeSelector
ā€¢ Annotations: 
JOB_KUBE_ANNOTATIONS
ā†’ 
global.jobs.kube.annotations
ā€¢ Image pull policy: 
JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_POLICY
ā†’ 
global.jobs.kube.mainContainerImagePullPolicy
ā€¢ Memory/CPU limits: 
JOB_MAIN_CONTAINER_MEMORY_LIMIT
ā†’ 
global.workloads.resources.mainContainer.memory.limit
[Helm chart V2 (Core); Configuring Airbyte] The knowledge sources do not provide a supported way to override the pod-level 
restartPolicy
for Airbyte job pods. If this is something you need, I'd recommend opening a GitHub issue or feature request with the Airbyte team. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
what does 
JOB_KUBE_TOLERATIONS
do @kapa.ai
k
JOB_KUBE_TOLERATIONS
defines one or more tolerations to be applied to Airbyte job pods (e.g., sync job pods) in Kubernetes. [Configuring Airbyte] Specifically: ā€¢ Tolerations are separated by 
;
 ā€¢ Each toleration is a set of 
k=v
pairs (some or all of 
key
, 
effect
, 
operator
, 
value
) separated by 
,
Example format:
Copy code
JOB_KUBE_TOLERATIONS: "key=airbyte-server,operator=Equals,value=true,effect=NoSchedule"
This allows job pods to be scheduled on nodes that have matching taints, enabling you to dedicate specific nodes to Airbyte workloads. [Configuring Airbyte] Helm Chart Equivalent In the Helm 
values.yaml
, the equivalent configuration is under `global.jobs.kube.tolerations`:
Copy code
global:
  jobs:
    kube:
      tolerations: []
[Values.yaml reference] > Note: As mentioned in a community forum post, if you omit the 
effect
field from the toleration string, the toleration may not be applied correctly. Any boolean values should be quoted to ensure they are passed through as strings. [Configuring Airbyte] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai what is the default for this? JOB_KUBE_TOLERATIONS
k
Based on the knowledge sources, the default value for 
JOB_KUBE_TOLERATIONS
is an empty string (
''
), meaning no tolerations are applied to job pods by default. This is confirmed in a community forum log entry showing:
Copy code
Using default value for environment variable JOB_KUBE_TOLERATIONS: ''
[Kubernetes - Bigquery sync failed] In the Helm 
values.yaml
, this is represented as an empty list:
Copy code
global:
  jobs:
    kube:
      tolerations: []
[Values.yaml reference] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
s
@kapa.ai my source container in the job pod exit for no reason
k
Based on the context of your earlier error trace, the source container exiting unexpectedly is likely not "for no reason" ā€” there are a few known causes documented in the knowledge sources: Known Reasons for Unexpected Source Container Exit 1. Root Cause: "Ran out of Input" on the companies stream As identified in your earlier trace, the primary failure was:
Copy code
Exception while syncing stream companies: Ran out of input
This caused the source process to crash, resulting in exit code 143. The exit was a consequence of this parsing/connection error, not a random termination. 2. OOM (Out of Memory Kill) With 15 GB of data being processed, the container may be getting killed by Kubernetes due to exceeding its memory limit. Exit code 137 indicates OOMKill, but exit code 143 can follow as a cascading effect. [OOM Exit Code 143] 3. Destination Failure Cascading to Source Your stacktrace shows a suppressed 
DestinationException
alongside the 
SourceException
. When the destination process exits with code 143, the orchestrator cancels the source process as well, causing it to also exit with 143. [Replication pod creation fails] 4. Concurrent Sync Resource Contention If you are running many syncs simultaneously, resource exhaustion (memory, ports, etc.) can cause pods to fail unexpectedly. [Running too many syncs concurrently] Recommended Next Steps 1. Check the full source container logs for any error before the exit ā€” the 
Ran out of input
error on the 
companies
stream is the most likely culprit in your case. 2. Verify your 
JOB_MAIN_CONTAINER_MEMORY_LIMIT
is sufficient for 15 GB+ syncs. 3. Try syncing the 
companies
stream in isolation with a smaller data range to reproduce and isolate the issue. [Sync Jobs Failing] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
13 Views
Open in Slack
PreviousNext
Powered by