@kapa.ai We're using WAL replication with a postgres database but are finding missing records. For example the latest record available for a specific table does not match what we have in the production database. I can confirm the issue is nothing to do with latency, the change occurred a while ago.

Missing records with Postgres CDC (WAL) replication in Airbyte can have several root causes. Here are the most likely explanations and fixes: 1. Full Refresh Triggered Instead of Incremental CDC If you see the log message: >

Saved offset is before Replication slot's confirmed_flush_lsn, Airbyte will trigger sync from scratch

This means Postgres purged the WAL segments Airbyte needed before they were consumed. This can happen when: • There are many database updates filling up the

pg_wal

directory, causing Postgres to purge WAL files. • The connector acknowledged WAL records, but the destination failed to consume them. Fixes: • Sync more frequently so WAL isn't purged between jobs. • Increase

wal_keep_size

on your Postgres server. [Postgres Troubleshooting] 2. WAL Disk Consumption / LSN Not Advancing For low-transaction-volume tables, the

confirmed_flush_lsn

may not advance as expected, causing syncs to fail or miss changes. [Postgres Troubleshooting] Fix — add a heartbeat table:

CREATE TABLE airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL DEFAULT current_timestamp,
    text TEXT
);

Then add it to your publication:

ALTER PUBLICATION <publicationName> ADD TABLE airbyte_heartbeat;

And configure the

Debezium heartbeat query

property in the connector:

INSERT INTO airbyte_heartbeat (text) VALUES ('heartbeat')

3. PgJDBC Regression (Connector v3.6.19+) A known regression in PgJDBC 42.7.0+ (used in connector v3.6.19+) causes

confirmed_flush_lsn

to advance automatically even when there are no new events for publication tables. This can cause the connector to skip changes. [Github Issue] Immediate workaround: Downgrade to connector version 3.6.18. 4. Publication/Catalog Mismatch If tables are in your Postgres publication but not in the Airbyte catalog (or vice versa), transaction records from those tables won't be synced or cleaned up from the WAL. Verify your publication matches your Airbyte catalog:

SELECT * FROM pg_publication_tables;

5. Missing Deletions If the missing records are specifically deletions, note that CDC records produced by

DELETE

statements only contain primary keys — all other data fields are unset. Also, changes made using `TRUNCATE`/`ALTER` will not appear in WAL logs and therefore won't be replicated. [Postgres Troubleshooting] To narrow down the issue, check your sync logs for any of the error messages mentioned above, and verify your replication slot status with:

SELECT * FROM pg_replication_slots;

This will show whether

confirmed_flush_lsn

is advancing as expected. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.