Hi all, running into a persistent CDC issue with

source-postgres:3.7.2

and hoping someone has seen this. Problem: After a full refresh completes successfully, the very next incremental CDC sync loads 0 records, and this repeats every time. Root cause (probably): The full refresh does the ctid snapshot correctly but saves the initial WAL LSN as its final state (the LSN at snapshot start, not at snapshot end). So on the next incremental run,

confirmed_flush_lsn == saved_offset_lsn

, Debezium sees itself as caught up, skips the snapshot, streams only heartbeats, and exits with 0 records. All WAL changes between snapshot start and the next sync run are lost. This seems related to

flush.lsn.source = false

, the slot LSN never advances during the ctid load so the saved state never moves forward. Question: Is this a known bug in 3.7.x? Is there a patched version or a recommended workaround short of doing a manual refresh every day? @kapa.ai

Yes, this is a known and well-documented bug in the

source-postgres

connector. Here's a summary of what's known: Root Cause The issue has two contributing factors, both introduced in version 3.6.19: [CDC LSN Bug] 1. PgJDBC regression: Starting from

postgresql:42.7.0

(pulled in via

datastore-postgres

from v3.6.19+), PgJDBC automatically advances

confirmed_flush_lsn

even when there are no new events for the publication tables — but the WAL still contains changes for tables outside the publication. [PgJDBC Issue] 2.

PostgresCdcCtidInitializer

captures

pg_current_wal_lsn()

multiple times during sync initialization, leading to a mismatch where the committed LSN is ahead of the saved state LSN. [Wrong LSN Commit] The net result is exactly what you're seeing:

saved_offset < confirmed_flush_lsn

, so subsequent syncs either fail or silently re-sync from scratch. Fix Status A fix was released in version

3.7.2

according to one community report, and separately a fix was noted in

3.6.34

(via PR #60876). [CDC LSN Bug] [Wrong LSN Commit] However, since you're already on 3.7.2 and still seeing the issue, it's possible the fix doesn't cover all scenarios (e.g., the zero-records incremental case). Workarounds (while waiting for a full fix) 1. Downgrade to

3.6.18

or

3.6.15

— confirmed by multiple community members to not exhibit this bug. [CDC LSN Bug] 2. Set up a Debezium heartbeat table — add a table to your publication and configure a heartbeat query so there's always at least one WAL event for the publication after the snapshot completes:CREATE TABLE public.airbyte_heartbeat ( id SERIAL PRIMARY KEY, timestamp timestamptz DEFAULT CURRENT_TIMESTAMP NOT NULL, text TEXT ); ALTER PUBLICATION airbyte_pub ADD TABLE public.airbyte_heartbeat;Then in the Airbyte source config under Advanced → Debezium heartbeat query, set:INSERT INTO public.airbyte_heartbeat (text) VALUES ('heartbeat');This ensures CDC doesn't break when there are no other changes to sync. [CDC LSN Bug] 3. Change "Invalid CDC position behavior" to

Re-sync data

— this avoids hard failures but will trigger a full refresh when the state is broken, which may not be viable for large tables. [CDC LSN Bug] 4. Manually update the connection state to the current `confirmed_flush_lsn`:SELECT confirmed_flush_lsn FROM pg_replication_slots WHERE slot_name = 'airbyte_slot';Note: this is a temporary fix and the issue tends to recur. [CDC LSN Bug] Given that you're on 3.7.2 and still hitting this, the heartbeat table approach is likely your most reliable workaround until a comprehensive fix lands. You may also want to follow issue #55846 directly for updates from the Airbyte engineering team. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

^ This is inaccurate. Both of the listed issues have been fixed. Please discuss with me in the other thread.