I have a MySQL v3.52.3 source that I use to sync d...
# airbyte-agents-ask-ai
t
I have a MySQL v3.52.3 source that I use to sync data from AWS MySQL RDS to BigQuery. I need to change the source RDS from the primary RDS instance to the replica RDS instance. But I cannot afford a full snapshot resync of all the tables. I have GTID enabled on both - primary and replica instances. I tried changing the source hostname from the primary to replica but the syncs failed with error-
Copy code
Stack Trace: io.airbyte.cdk.ConfigErrorException: Incumbent CDC state is invalid, reason: java.lang.NullPointerException: Cannot invoke "io.debezium.connector.mysql.gtid.MySqlGtidSet$UUIDSet.getUUID()" because "other" is null
I tried changing the server_id in the airbyte connection manually to point to the replica's server_id - but still got the same error. While looking for solutions online, I found this open issue on airbytehq - https://github.com/airbytehq/airbyte/issues/75558 What is the suggested way for me to shift the sync source from primary to replica instance? And if the above issue is the fix - how soon can I expect it to go live?
k
MySQL CDC GTID Error When Switching to Replica The error you're seeing is a known bug in the
source-mysql
connector. When the MySQL server's GTID topology changes (which happens when you switch from primary to replica, since the replica has a different server UUID), Debezium's
MySqlGtidSet.subtract()
throws a
NullPointerException
because the saved CDC state contains UUIDs not present in the new server's GTID set. [Github Issue] Root Cause The saved CDC state references the primary's server UUID, but the replica has a different UUID. Debezium 3.1.x (used in older connector versions) doesn't handle this UUID mismatch gracefully and throws an NPE. [Github Issue] Current Workaround (Without Full Resync) Unfortunately, the knowledge sources do not describe a way to switch from primary to replica without triggering a full CDC resync in this scenario. The only workaround mentioned is: > Reset the connection state via the Airbyte UI (Connection → Settings → "Clear data and reset" for affected streams) to force a fresh CDC re-sync. [Github Issue; CDC Discussion] This would require a full snapshot, which you've stated you cannot afford. Manually changing the
server_id
alone is not sufficient because the GTID sets themselves differ between primary and replica. Fix Status Several PRs have been opened to address this: 1. PR #75559 — Bumps Debezium from
3.1.2.Final
to
3.4.2.Final
in the Bulk Extract CDC toolkit, which contains the upstream null-safe GTID handling fix. 2. PR #75560 — Aligns
source-mysql
to the updated Bulk CDK version. 3. PR #75530 / PR #75529 — Add defensive
try-catch
around GTID comparison in
validate()
, routing NPEs through
abortCdcSync()
which respects your
invalidCdcCursorPositionBehavior
setting. 4. PR #79182 — Another fix specifically for the warm-start NPE, also adding
try-catch
with support for
RESET_SYNC
behavior. However, even with these fixes applied, the behavior when catching the NPE is to either fail with an actionable error or auto-reset and re-snapshot (depending on your
invalidCdcCursorPositionBehavior
setting) — not to seamlessly continue CDC from the replica's GTID position. [PR #79182] Regarding Timeline The knowledge sources do not contain release date information for when these fixes will be generally available. Since you're on v3.52.3, I'd recommend: • Checking the MySQL connector changelog to see if any of the above PRs have been included in a release newer than your current version. • If you're on Airbyte Cloud, opening a support ticket referencing issue #75558 to get guidance specific to your situation. • Joining the Airbyte Community Slack for peer support. Summary I'm sorry there isn't a cleaner path here — the fundamental issue is that the replica's GTID history differs from the primary's, and the connector currently has no mechanism to reconcile that without a fresh snapshot. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
t
@kapa.ai how can I raise a feature request for this?
k
To raise a feature request for this, you can start a GitHub Discussion in the Airbyte repository: 1. Go to GitHub Discussions 2. Start a new discussion describing your need — specifically, the ability to switch MySQL CDC source from primary to replica without requiring a full resync when GTID is enabled on both instances. This is the recommended path for requesting new data replication features. [Issues and Requests] If you are an Airbyte Cloud customer, you can also raise it directly via the Support Center, referencing the existing issue #75558 for context. [Issues and Requests] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.