@kapa.ai Hey — running into a weird MongoDB CDC issue on self-hosted Airbyte (v1.8.3, source-mongodb-v2) and hoping someone's seen this before. We had 21 MongoDB source connections (single-database CDC, 30-min sync schedule) that all got recreated at once after a reset. They ran fine for a while, then started failing with

Saved offset is not valid

/ oplog history lost errors — but the timing was weird: it failed almost exactly when our configured oplog retention ran out. We bumped retention from 24h to 72h thinking that'd fix it... and it just failed again 3 days later, same error. So we dug into the actual sync logs, and found the saved resume token's timestamp was ~72 hours old — meaning the checkpoint hadn't moved forward at all in ~144 sync runs, even though those syncs were reporting success. Bumping retention just delayed the same failure, it didn't fix anything. We ruled out a connector version bug (checked 2.0.2 vs 2.0.7, both pin the exact same CDK and Debezium versions, so no code difference there). Best guess right now is something about Debezium's heartbeat not properly refreshing the saved offset for quiet/low-write databases, especially since Airbyte restarts a fresh Debezium process every single sync instead of running it continuously. Has anyone hit this — offset going stale in roughly the same time as your oplog retention no matter what you set it to, especially after mass-recreating a bunch of CDC connections at once? Trying to figure out if this is a known Debezium/Airbyte interaction or something specific to our setup.

Thinking completed Yes, this is a known behavior and issue when using CDC with

source-mongodb-v2

on low-activity collections or databases [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.]. What is Happening 1. Failure matches the oplog window exactly: When an incremental CDC sync runs, the connector starts reading from the last saved MongoDB change stream resume token. If no new events or qualifying writes occur in the monitored streams during that sync window, the connector may fail to advance its committed resume token [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.; Github Issue (open): [source-mongodb-v2] CDC resume token timestamp frozen — incremental sync replays 101 records/run forever (initial_waiting_seconds workaround from 71046 ineffective)]. As an Airbyte contributor noted in an issue discussing this behavior, if the sync does not read change records to advance the LSN / resume token before the oplog recycles, MongoDB eventually purges that point in history. Once MongoDB's oplog retention window passes (24 hours, 72 hours, etc.), the saved resume token falls outside the oplog, triggering a

ChangeStreamHistoryLost

/

ChangeStreamFatalError

and Airbyte’s

[config_error] Saved offset is not valid

error [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.; Github Issue (closed): Saved offset is not valid for MongoDB incremental sync]. 2. Debezium Heartbeat and Batching Behavior: Airbyte runs connectors as ephemeral container jobs rather than continuous daemon processes. Even though Debezium has a heartbeat interval property configured (

heartbeat.interval.ms = 10000

) [MongoDbCdcProperties.java], known issues document cases where the resume token timestamp remains frozen or fails to advance to real time when there is no data movement or when batches end prematurely [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.; Github Issue (open): [source-mongodb-v2] CDC resume token timestamp frozen — incremental sync replays 101 records/run forever (initial_waiting_seconds workaround from 71046 ineffective)]. 3. Check for Legacy Multi-State Bug (Connector < 2.0.4): In versions prior to

2.0.4

,

source-mongodb-v2

had a known bug where the connection state saved two different resume tokens under `shared_state`:"shared_state": { "state": { "[\"prod\",{\"server_id\":\"prod\"}]": resume_token, "[mongo_url]": resume_token } }One token would update while the other stayed permanently pinned to the initial sync time. Once the oplog window expired, the stale token caused validation to fail [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.]. This specific bug was fixed in connector version

2.0.4

[Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.]. If your connection was initialized on an earlier connector version, check your connection state in Airbyte to verify whether multiple state keys are present. Workarounds • Ensure Ongoing Writes in Monitored Streams: Because the connector relies on incoming change stream records to advance the resume token, Airbyte guidance states that it is necessary to have data movement or include an active table/collection in the CDC sync so the cursor advances and does not get dropped by the oplog retention window [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.]. Similar to how a heartbeat table is used for PostgreSQL CDC [Github Issue (open): [source-postgres] CDC sync is making the confirmed flush LSN to a value saved state LSN>], generating regular dummy updates or heartbeats on a monitored collection ensures events exist for the connector to read and advance the offset before the oplog retention expires. • Upgrade the Connector: If you are running versions prior to

2.0.4

, upgrade the

source-mongodb-v2

connector to

2.0.4

or newer to ensure you are not hitting the duplicate-state bug [Github Issue (closed): [source-mongodb-v2] Failed syncs due to [config_error] Saved offset is not valid.]. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.