https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> I'm getting the following error whe...
# replication-troubleshooting
b
@kapa.ai I'm getting the following error when syncing a connection after upgrading to 1.9.2:
Copy code
2025-12-10 12:09:50,703 [io-executor-thread-2]    ERROR    i.a.c.s.e.h.IdNotFoundExceptionHandler(handle):33 - Not found exception class NotFoundKnownExceptionInfo {                                                                                                                                               │
│     id: null                                                                                                                                                                                                                                                                                                        │
│     message: Id not found: Could not find attempt stats for job_id: 12483 and attempt no: 0                                                                                                                                                                                                                         │
│     exceptionClassName: io.airbyte.commons.server.errors.IdNotFoundKnownException                                                                                                                                                                                                                                   │
│     exceptionStack: [io.airbyte.commons.server.errors.IdNotFoundKnownException: Id not found: Could not find attempt stats for job_id: 12483 and attempt no: 0,     at io.airbyte.commons.server.errors.handlers.IdNotFoundExceptionHandler.handle(IdNotFoundExceptionHandler.kt:32),     at io.airbyte.commons.ser │
│     rootCauseExceptionClassName: java.lang.Class                                                                                                                                                                                                                                                                    │
│     rootCauseExceptionStack: [io.airbyte.commons.server.errors.IdNotFoundKnownException: Could not find attempt stats for job_id: 12483 and attempt no: 0,     at io.airbyte.commons.server.handlers.AttemptHandler.getAttemptCombinedStats(AttemptHandler.kt:251),     at io.airbyte.server.apis.controllers.Attem │
│ }
The error occurs in the 
airbyte-abctl-server
pod
k
That error means the API is trying to read attempt statistics for a job/attempt that exist in the job metadata, but there is no corresponding stats record in persistence: > 
Id not found: Could not find attempt stats for job_id: 12483 and attempt no: 0
This is the same underlying condition that produces the “Missing stats for job … attempt …” warnings you see in your server logs. [[Job stats warning](https://discuss.airbyte.io/t/4016)] In that thread, the connection executions themselves were fine; the problem was only when the API tried to aggregate or display stats for those attempts. From the information available in the knowledge sources: - There is no explicit fix or migration step documented for this “missing attempt stats” condition on 1.9.x. - A related 0.40.x issue (
IdNotFoundKnownException
after upgrade) was due to **skipping minor versions during upgrade**; re‑running the upgrade sequentially across intermediate versions fixed the object-not-found condition in that case. [[0.40.4 upgrade issue](https://discuss.airbyte.io/t/2471)] - Another user saw “missing stats for job …” logs after changing log/state storage configuration (e.g., MinIO → GCS), but the connections ran successfully; the logs were just noisy, and there was no documented corrective action beyond investigating config changes. [[Job stats warning](https://discuss.airbyte.io/t/4016)] Given that: 1. The knowledge sources do not provide a concrete remediation for 1.9.2’s 
AttemptHandler.getAttemptCombinedStats
failures. 2. They only suggest that: - These errors happen when the stats row for a given job/attempt is missing. - Storage/config changes or incomplete migrations can lead to related “not found” issues. Because of this limitation, I can’t reliably tell you how to fix it in 1.9.2 (e.g., whether you need to re-run a migration, backfill stats, or adjust configuration), beyond the general idea that a mismatch exists between job attempts and their stats records. To move forward: - If you skipped versions when upgrading to 1.9.2, the 0.40.x thread suggests re-running the upgrade sequentially through intermediate versions can resolve related “object not found” problems. [[0.40.4 upgrade issue](https://discuss.airbyte.io/t/2471)] - If you recently changed storage settings (logs, state, etc.), double‑check those changes; a prior case started seeing “missing stats” messages immediately after altering storage configuration. [[Job stats warning](https://discuss.airbyte.io/t/4016)] But the knowledge sources do not contain a documented, version‑specific fix for your exact 1.9.2 
airbyte-abctl-server
error.
b
When diving deeper, it seems that no connection pod is starting when I start the sync in the airbyte UI @kapa.ai
k
A message from kapa.ai
The symptoms you’re seeing (no connection/worker pods start when you trigger a sync, plus “missing attempt stats” / 
IdNotFoundKnownException
) match known cases where: 1. Jobs are created in the control plane, but 2. The actual worker pods (orchestrator/source/destination) are not launched, so 3. No attempt stats are ever written, and the API/UI later fails when trying to read them. A similar Kubernetes case on 2.0 reports jobs that either run or complete with 
0 bytes|no records loaded
, while no orchestrator/source/destination pods are started for the failing runs. [[Jobs stuck, no pods](https://github.com/airbytehq/airbyte/issues/68115)] Another thread shows the same “missing stats for job … attempt …” pattern at the server level during listing/aggregation of job history. [[Job stats warnings](https://discuss.airbyte.io/t/4016)] However, in the available sources there is: - No explicit root cause or fix documented for: - 1.9.2 specifically, or - the 
airbyte-abctl-server
AttemptHandler.getAttemptCombinedStats
error you pasted. - Only confirmations that: - “Missing stats for job … attempt …” means the stats rows are absent when the API tries to read them. [[Job stats warnings](https://discuss.airbyte.io/t/4016)] - In a separate upgrade scenario (
0.40.0 → 0.40.4
), an 
IdNotFoundKnownException
after upgrade was related to migrations / skipped versions, but the exact remediation steps are not detailed in the sources you shared. [[0.40.4 upgrade issue](https://discuss.airbyte.io/t/2471)] Because of these gaps, I can’t reliably tell from the knowledge sources why your worker pods are not starting on 1.9.2, nor give a concrete, version‑specific fix. What the sources do support you checking: - Upgrade / migration path – similar “id not found” issues have been tied to upgrade/migration problems and inconsistent metadata. [[0.40.4 upgrade issue](https://discuss.airbyte.io/t/2471)] - Control‑plane vs. data‑plane linkage – another user on Kubernetes + 2.0 verified that 
workspace
and 
dataplane_group
tables were correctly linked when jobs randomly failed to start pods. [[Jobs stuck, no pods](https://github.com/airbytehq/airbyte/issues/68115)] - Any recent storage / env changes – log/state storage changes (e.g., MinIO → GCS) coincided with persistent “missing stats for job …” messages, even when syncs completed. [[Job stats warnings](https://discuss.airbyte.io/t/4016)] Given the limitations of the provided material, I’m sorry I can’t point to a documented 1.9.2‑specific fix for “no connection pods starting” plus 
IdNotFoundKnownException
in 
airbyte-abctl-server
.
b
@kapa.ai I have looked into the workloads table in the airbyte database and I'm seeing that the workloads have been terminated with the following message: 
Airbyte could not start the process within time limit. The workload was never claimed.
k
A message from kapa.ai
That database message: > 
Airbyte could not start the process within time limit. The workload was never claimed.
means the Workload Launcher never successfully claimed the workload, so the pod to run the sync/check was never started, and after a timeout Airbyte marked the workload as terminated. From the Workloads architecture: - A job becomes a workload, which is put in a queue. - The Workload Launcher periodically calls the Workload API to claim workloads and then launch Kubernetes pods for them. [[Workloads & Jobs](https://docs.airbyte.com/platform/understanding-airbyte/jobs)] - If a workload is never successfully claimed, no connector pod is created, and the workload stays in a queued/launched‑but‑inactive state until timing out. In a related Kubernetes case on 2.x, the launcher logs repeatedly showed:
Copy code
text WorkloadApiClient(claim): Claimed: false for workload ...
and the launcher then skipped all subsequent stages (including status checks), so pods were not started and workloads never transitioned out of the “launched”/queued state. [[Jobs stuck, no pods](https://github.com/airbytehq/airbyte/issues/68115)] This is the same class of symptom you’re seeing: “workload was never claimed” ⇒ no pods ⇒ timeout message recorded in the DB. However, the knowledge sources: - Do not explain, for 1.9.2 specifically, why the claims are failing in your case. - Do not document a concrete fix for “workload was never claimed” timeouts (beyond general discussion of claim failures and stuck workloads). [[Jobs stuck, no pods](https://github.com/airbytehq/airbyte/issues/68115)] So, based on what’s available: - Your finding in the 
workload
table is consistent with the observed behavior: no connection pods start because the workloads are never claimed by the launcher within the time limit. - The exact root cause and remediation for this on 1.9.2 are not covered in the provided sources, so I can’t reliably tell you how to fix it (e.g., whether it’s configuration, an upgrade/migration problem, or a product bug).
b
error_logs.txt
error_logs.txt
Untitled.txt
Untitled.txt
@kapa.ai I've managed to get my sync to run now, but I'm getting another error at the end of the replication job.I've attached a snippet with the full log of the replication job. Any idea what is going on?
k
A message from kapa.ai
Based on the sources you shared, there are several reports of replication jobs completing most of the work successfully, then erroring at the very end, causing the attempt to be marked as failed even though data was written: - Snowflake / custom source on Kubernetes: data loaded, tmp tables cleaned, then 
java.lang.RuntimeException: Cannot find pod while trying to retrieve exit code. This probably means the Pod was not correctly created.
[[RuntimeException exit code](https://discuss.airbyte.io/t/2421)] - Multiple cases where the destination (or orchestrator) closes with errors like 
Destination has not terminated. This warning is normal if the job was cancelled.
or “Error closing resource …; recording failure but continuing”, again after most work is done. [[Sync failed but data ok](https://github.com/airbytehq/airbyte/issues/57000); [Firestore close error](https://github.com/airbytehq/airbyte/issues/70837); [Custom dest close error](https://github.com/airbytehq/airbyte/issues/69138)] In these situations, the pattern is: 1. Replication runs, streams finalize, data appears in the destination. 2. At the teardown/close phase, Airbyte fails to: - Retrieve pod exit code, or - Close the destination/orchestrator cleanly. 3. The worker then throws a 
WorkerException
/ 
RuntimeException
, and: - The attempt is marked as FAILED in the UI, - But the data may already be present and valid. However, the knowledge sources you provided: - Do not include your specific 1.9.2 log snippet, nor any 1.9.2‑specific post‑replication error with an official diagnosis. - Do not document a clear, general fix for these “end‑of‑job” failures (they’re acknowledged as issues, but the remediation is not detailed in the shared materials). [[RuntimeException exit code](https://discuss.airbyte.io/t/2421); [Sync failed but data ok](https://github.com/airbytehq/airbyte/issues/57000)] So all I can say with confidence, grounded in the sources, is: - Errors at the very end of replication that mark the job as failed, even when data appears loaded, are a known class of issues (usually around pod lifecycle / exit code handling / close phase). - For your exact error and version, the knowledge sources here do not provide enough information to say definitively what is going on or how to fix it. If you can share the exact error line(s) from the end of your replication log (especially the first `ERROR`/`WorkerException` stacktrace), I could try to match it more precisely to one of the known patterns above; but with the current material, I can only relate it to this general class of “successful replication, failed teardown” issues, without a documented 1.9.2‑specific solution.
4 Views
Open in Slack
PreviousNext
Powered by