<@U04S8PLJWMT> We have migrated our airbyte instan...
# replication-ask-ai
p
@kapa.ai We have migrated our airbyte instance to 2.0.1 (from 1.5.1) - we are using helm to deploy the instance in k8s. After the upgrade, we seem to be unable to trigger any source check (stuck with the below stack trace in the worker component). Any know issue with this?
Copy code
2025-12-15 14:44:27,904 [workflow-method-connection_manager_5611cad0-f3de-4366-a9cd-0ea1afe...-f74c0650-6d6b-4054-a2df-b414213cbf75]       INFO    i.a.w.t.s.ConnectionManagerWorkflowImpl(checkConnectionsWithCommandApi):650 - SOURCE CHECK: Starting
2025-12-15 14:44:27,914 [workflow-method-connection_manager_5611cad0-f3de-4366-a9cd-0ea1afe...-f74c0650-6d6b-4054-a2df-b414213cbf75]       INFO    i.a.w.t.s.ConnectionManagerWorkflowImpl(runCheckWithCommandApiInChildWorkflow):1071 - Running command check for source with id fa1c844f-939f-4b31-b6e3-0c01748ef162 with the use of the new command API
k
There are a few relevant, but only partial, hints in the provided sources. 1. Similar “checks hang / don’t start” behavior after upgrades - One user on 2.0 reported that syncs either hang or complete with
0 bytes|no records loaded
, and no orchestrator/source/destination pods are launched. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] - In that case, the root cause was dataplane mis‑wiring: two rows existed in
public.dataplane_group
(one pre‑2.0, one post‑2.0), and jobs were being pushed to dataplane A while the workload launcher was listening to dataplane B. Updating the
dataplane_group_id
for the workspace to the correct one fixed the issue. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] Given your symptoms (“after the upgrade, unable to trigger any source check” and logs stuck at
SOURCE CHECK: Starting
), this dataplane mismatch is a strong candidate. 2. General guidance on upgrades The official docs only describe the Helm upgrade flow and don’t mention a known 2.0.1‑specific bug that matches your stack trace. [[Upgrading Airbyte – platform](https://docs.airbyte.com/platform/operator-guides/upgrading-airbyte); [Upgrading Airbyte – next](https://docs.airbyte.com/platform/next/operator-guides/upgrading-airbyte)] They also note that connectors may be upgraded automatically or may need manual updating, but nothing directly about checks hanging. 3. Other issues around checks failing/hanging - There are issues where checks fail or hang with different root causes (e.g., specific connector versions, or 422 errors when updating sources), but these don’t closely match your “post‑2.0.1, all checks stuck at start” behavior. [[All sync jobs hanging](https://github.com/airbytehq/airbyte/issues/55812); [[source-mysql] unable to pass check](https://github.com/airbytehq/airbyte/issues/52075); [source-google-ads 422](https://github.com/airbytehq/airbyte/issues/60900)] - None of those explicitly identify a 2.0.1‑wide bug with
checkConnectionsWithCommandApi
. Because of that, I can’t point to a documented “known issue with 2.0.1 checks” in the sources. The only concrete, similar post‑2.0 upgrade problem with jobs not progressing is the dataplane mismatch. ### What you can check (based on the known workaround) In your Airbyte database (Postgres): 1. List dataplane groups:
Copy code
sql    SELECT * FROM public.dataplane_group;
2. Check which dataplane group each workspace is using:
Copy code
sql    SELECT id, organization_id, dataplane_group_id    FROM public.workspace;
3. If you see multiple
dataplane_group
rows (e.g., one legacy, one new) and your active workload launcher is associated with a different
dataplane_group_id
than your workspace, align them by updating the workspace’s
dataplane_group_id
to the correct one (the workaround described by another user). [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] If dataplane configuration is already consistent, the knowledge sources don’t provide another confirmed 2.0.1‑specific root cause for this log pattern, so I can’t reliably suggest a different internal fix beyond: - Verifying the upgrade followed the documented Helm steps. [[Upgrading Airbyte – platform](https://docs.airbyte.com/platform/operator-guides/upgrading-airbyte)] - Ensuring the worker and workload launcher pods are healthy and logs don’t show dataplane or command‑API‑related errors (not covered in the provided docs/issues). If you can confirm whether you have multiple
dataplane_group
rows and what your
workspace.dataplane_group_id
is, that would be the first concrete thing to validate based on the available information.
p
@kapa.ai ok that got fixed, but now it seems that crons / schedules are not running
k
A message from kapa.ai
The most relevant info in the provided sources about “jobs not running after upgrade” is around the dataplane / scheduler behavior in Airbyte 2.x. After fixing the dataplane mapping so manual checks/syncs work, but scheduled jobs (cron / “every X hours”) still don’t run, the closest documented behavior is: - After upgrading to 2.0, some users saw that no new job pods were launched, even though connections existed and UI looked fine. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] - Root cause there was again dataplane wiring: 2 entries in
public.dataplane_group
(old vs new), and jobs being pushed to one dataplane while the workload launcher listened to another. Aligning the
workspace.dataplane_group_id
to the active dataplane group fixed jobs starting at all. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] You’ve already corrected a dataplane issue so that manual checks now work, which means: - Temporal, workers and workload‑launcher are at least partially healthy. - The remaining issue is specifically with scheduling. From the docs, Airbyte’s scheduler guarantees: - Syncs will run either on a fixed interval (“Scheduled” every N hours) or via CRON (Quartz‑style cron, with a timezone). [[Sync schedules](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-schedules)] - Only one sync per connection at a time; if one is running, the next scheduled one waits. - Schedule accuracy is +/- 30 minutes. [[Sync schedules](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-schedules)] However, none of the provided issues or docs explicitly describe a 2.0.1‑specific bug where cron/schedules stop running while manual checks still work. The known dataplane problems affected all jobs (manual and scheduled) until fixed. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115); [No jobs starting after upgrading to version 2](https://github.com/airbytehq/airbyte/issues/68196)] Given the limits of the sources, I can’t point to a documented “cron scheduler broken in 2.0.1” issue. Within what is documented, things you can verify: 1. Dataplane consistency for all workspaces Make sure the workaround you applied is consistent across all rows:
Copy code
sql    SELECT * FROM public.dataplane_group;    SELECT id, organization_id, dataplane_group_id FROM public.workspace;
All workspaces that should be running scheduled jobs need to point to the same, active dataplane group used by the workload launcher. The cited workaround only mentions updating
workspace.dataplane_group_id
and doesn’t distinguish between manual vs scheduled jobs. [[airbyte services stuck](https://github.com/airbytehq/airbyte/issues/68115)] 2. Schedule type and timing From the UI/API, confirm that the connections are still set to
Scheduled
or
Cron
and not
Manual
, and that the cron expression/timezone or interval are correct. [[Sync schedules](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-schedules)] 3. No overlapping “stuck” runs Since only one sync per connection can run at a time and scheduled runs that collide will wait until the previous finishes, confirm there are no long‑running / stuck jobs blocking subsequent schedules. [[Sync schedules](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-schedules)] Because the sources don’t give more detail on this exact situation, I can’t reliably suggest a deeper internal fix specific to 2.0.1. If: - Dataplane group wiring is now correct for all workspaces, and - Connections are configured as Scheduled/Cron with valid expressions, and - No old jobs are stuck “Running”, then the behavior you’re seeing isn’t covered in the provided docs/issues, and you’d likely need to share Temporal / cron / workload‑launcher logs upstream (e.g., under the existing 2.x scheduler/dataplane issues) for further investigation.