@kapa.ai. For some reason I'm missing updates from RDS Mysql using CDC. I'm using the open-source version in k8s and I'm ingesting data into redshift.

Missing CDC Updates from RDS MySQL to Redshift This is a known issue with several potential causes. Here are the most common ones and how to address them: 1. Binlog Expiring Too Quickly If your RDS binlogs are purged before Airbyte can read them, the connector will miss changes — or trigger a full refresh with the log message: >

Saved offset no longer present on the server, Airbyte is going to trigger a sync from scratch

Fix: Increase binlog retention on RDS:

call mysql.rds_set_configuration('binlog retention hours', 24);

A value of at least 24–72 hours is recommended. [MySQL Troubleshooting] 2. MissingTableMapEventException A common error after the initial snapshot is:

MissingTableMapEventException: No TableMapEventData has been found for table id: XXXXX

This typically happens when the CDC stream starts reading binlogs mid-event-group, often caused by dropped network connections between your K8s cluster and RDS. The repeated

Aborted connection

warnings in RDS logs are a strong indicator of this. [CDC Sync Failures] Fix: Check your K8s network configuration — Security Groups, Network ACLs, and NAT Gateway settings — to ensure long-lived TCP connections to RDS are not being severed. 3. Changes During Initial Snapshot Window Data changes (inserts/updates) that occur between the start and completion of the initial full-refresh snapshot may not be captured in the first incremental sync. [Missing Rows Issue] This is a known limitation of the snapshot-to-streaming transition. 4. Insufficient CDC Initial Wait Time If the connector doesn't have enough time to process the change logs, it may miss events. Fix: Increase the

Checkpoint Target Time Interval

in the source configuration (valid range: 300–1200 seconds). [CDC Wait Time] 5. Transient Binlog Deserialization Errors Intermittent

EOFException

or

EventDataDeserializationException

errors during binlog reading can cause syncs to fail. A recent fix (source-mysql

3.53.1

) reclassifies these as transient errors so they are automatically retried instead of surfacing as config errors. [CDC Error Fix] Fix: Ensure you're on the latest version of the MySQL source connector. Recommended checklist: • ✅ Set

binlog retention hours

≥ 24 on RDS • ✅ Verify network connectivity between K8s and RDS (check for aborted connections in RDS logs) • ✅ Update to the latest MySQL source connector version • ✅ Review sync logs for specific error messages to narrow down the root cause 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.