I’m extracting data from MS SQL Server database us...
# replication-troubleshooting
r
I’m extracting data from MS SQL Server database using CDC. I have three connections setup for this database. Connection 1 is pulling a subset of tables on a cron schedule at 10 minute intervals. Connections 2 and 3 are pulling different subset of tables at 1 hour and 24 hour intervals, respectively. Connection 1 has been running without any problems for over a month. Connections 2 and 3 are generating the following error in the log: Interrupted while snapshotting table <some table>. The table that connections 2 and 3 are failing on is not one of the tables selected in the sync schema. Here is some info and troubleshooting I’ve tried. • Source Microsoft SQL Server (MSSQL) 4.2.5 • Destination Snowflake 3.15.10 • SQL Server Database SQL Server 2014 SP3 • Database has CDC configured for required tables • Database has Snapshot Isolation turned on • I paused all other streams in our Airbyte instance in case this was a resource constraint • Airbyte instance is open source, version 2.0.1 • I’ve tried deleting the connection and recreating it multiple times • I’ve also tried dropping the table that the snapshot gets interrupted on, in which case the snapshot will then get interrupted on a different table • Connections 2 and 3 will complete the initial load, but the following line remains in the connection state. “{\“commit_lsn\“\“0000c1140003d1180003\“,\“snapshot\“true,\“snapshot_completed\“:true}” • The connection state and the error in the log indicates to me that it never successfully completes the initial snapshot. So the connection will run once refreshed, and will continue to run successfully for 72 hours until the snapshot LSN is cleared from the source DB, at which point the connection begins failing. 72 hours is our CDC retention window on the source db. I could use some assistance in troubleshooting this connection error. We have ~30 dbs each with multiple connections, and I’m getting the error outlined above on approximately five of them and I can’t seem to find any rhyme or reason as to why, especially when one connection to the database works with no issues, but another connection to the same database doesn’t seem to complete the initial snapshot successfully. Any help would be greatly appreciated. Thank you.
k
It appears that your connections 2 and 3 are stuck in a state where the initial snapshot is not being finalized correctly, which eventually leads to the "Saved offset no longer present on the server" error once the CDC retention period (72 hours) passes. Based on the knowledge sources, here are the potential causes and troubleshooting steps: ### 1. Snapshot LSN and Retention Window The log entry
“snapshot_completed“:true
suggests Airbyte believes the snapshot finished, but if the
commit_lsn
remains static and the connection later fails with a missing offset, it indicates the saved position is no longer in the SQL Server transaction logs [[Sync fails after CDC cleanup](https://github.com/airbytehq/airbyte/issues/68150)]. When the CDC cleanup job runs on the MS SQL server, it removes entries older than your 72-hour retention window. If the connector hasn't successfully transitioned from "snapshot" mode to "streaming" mode (which would update the LSN), the next sync attempt will look for an LSN that has been deleted, causing a failure [[Sync fails after CDC cleanup](https://github.com/airbytehq/airbyte/issues/68150)]. ### 2. Table Interference and Heartbeats You mentioned the snapshot is interrupted on tables not selected in the sync. In MS SQL CDC, the connector may still monitor change events for the entire database to maintain the LSN position. * Heartbeat Issues: There is a known issue where MS SQL CDC syncs can run indefinitely or hang if there is low activity on the tables being tracked, causing the LSN heartbeat to stop progressing [[MSSQL CDC Sync runs infinitely](https://github.com/airbytehq/airbyte/issues/60304)]. * Missing Activity: If the tables in Connections 2 and 3 have very low traffic, the connector might not receive enough events to commit a new state, preventing the replication slot/LSN from advancing [[Stack Syncs and Automated Full Resyncs](https://discuss.airbyte.io/t/2675)]. ### 3. Connection and Buffer Constraints Since Connection 1 (10-minute interval) works while the others (1-hour and 24-hour) fail, the frequency of the sync is likely the deciding factor. * Back Pressure: Larger syncs or syncs with many tables can cause back pressure from the destination (Snowflake), leading to timeouts during the snapshot phase [[MySQL to Snowflake Fails](https://discuss.airbyte.io/t/2991)]. * Recommended Adjustments: * Increase the sync frequency for Connections 2 and 3 to match Connection 1 (e.g., every 10-15 minutes). This ensures the LSN is acknowledged and updated frequently, well before the 72-hour cleanup job runs [[Sync fails after CDC cleanup](https://github.com/airbytehq/airbyte/issues/68150)]. * If you have a large number of tables (more than 30), consider splitting them into even more connections, as high table counts can lead to "interrupted" states due to buffer limitations [[Postgres CDC connector stuck](https://discuss.airbyte.io/t/4373)]. ### Summary of Recommendations: 1. Increase Sync Frequency: Change Connections 2 and 3 to run more frequently (e.g., every 15 minutes) to ensure the LSN is updated before it is purged from the server logs [[Sync fails after CDC cleanup](https://github.com/airbytehq/airbyte/issues/68150)]. 2. Verify CDC on New Tables: Ensure that any tables recently added to the database have CDC explicitly enabled at the database level [[CDC Limitations](https://docs.airbyte.com/platform/2.0/understanding-airbyte/cdc#limitations)]. 3. Reset and Re-sync: Because the current saved offset is already lost (or "stuck" in a snapshot state that refers to a deleted LSN), you must Reset the connection to trigger a fresh initial snapshot [[Sync fails after CDC cleanup](https://github.com/airbytehq/airbyte/issues/68150)]. The knowledge sources do not explicitly mention why a table not in the schema would trigger an "interrupted" log, but this often points to the underlying Debezium engine scanning the database's transaction logs for all CDC-enabled tables to find the correct starting point.
r
Anyone able to assist?