@kapa.ai, I am doing postgres to postgres cdc replication, but the issue is that incremental replication is not working it is not replicating incremental updates what could be the reason ?

There are several known reasons why Postgres CDC incremental replication may fail to capture updates. Here are the most common causes and fixes: 1. Connector/Platform Version Compatibility Issue A known compatibility bug exists where Postgres source connector versions `0.4.27`+ used with Airbyte versions

v0.38.3-alpha

through

v0.39.17-alpha

causes incremental state to not be updated, making connections always sync from the stored state cursor. [Forum Post] Fix: Upgrade the Postgres source connector to

0.4.31

or later, and ensure your Airbyte platform is on a compatible version. 2. WAL Resume Position Timeout If the connector times out while searching for the WAL resume position, incremental syncs will produce 0 records. [Github Issue] Fix: Increase the Initial Waiting Time in Seconds (Advanced setting) on the connector settings page. One user resolved this by setting it to

2400

. 3. WAL Files Purged by Postgres If WAL files needed for incremental sync have been removed by Postgres, Airbyte will fall back to a full refresh. You'll see this log message: >

Saved offset is before Replication slot's confirmed_flush_lsn, Airbyte will trigger sync from scratch

Fixes: [Postgres Troubleshooting] • Sync more frequently to prevent WAL accumulation. • Set a higher

wal_keep_size

in your Postgres config. 4. Table Not Added to Publication If the table is not included in the Airbyte publication, updates won't be captured. [Forum Post] Fix: Verify your table is in the publication:

SELECT * FROM pg_publication_tables;

5. CDC Requirements Not Met Ensure your setup meets these requirements: [Postgres Troubleshooting] • Tables must have primary keys (incremental CDC is not supported without them). • Data must be in tables, not views. • Changes must use `DELETE`/`INSERT`/`UPDATE` — `TRUNCATE`/`ALTER` changes are not captured in WAL. • Use Postgres v10 or above for CDC workflows. • Only one Airbyte destination per CDC source (multiple destinations will each receive only partial data). 6. Schema Mismatch (Debezium Type Conversion Error) A Debezium type conversion error can cause CDC incremental syncs to show 0 records, especially when processing DELETE operations. [Github Issue] Temporary workaround: Switch the affected stream to Full Refresh sync mode until the issue is resolved. To help narrow down the issue, check your sync logs for any of the error patterns above and share your connector version and Airbyte platform version if the problem persists. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai, there are no errors that it sliently stops incremental replication without any errors plus table is also inside the publication

Based on the knowledge sources, silent CDC failures with 0 records and no errors are a well-documented issue. Here are the most likely causes: 1. PgJDBC Regression (Connector >= 3.6.19) This is the most likely culprit for silent failures. Starting from connector version

3.6.19

, a PgJDBC regression (

42.7.0+

) causes

confirmed_flush_lsn

to automatically advance even when there are no new events for your publication tables — but the WAL still has changes from tables outside the publication. This causes the saved state offset to fall behind

confirmed_flush_lsn

silently. [Github Issue] Immediate fix: Downgrade to connector version

3.6.18

. [Github Issue] 2. Low Transaction Volume on Publication Tables If your publication tables have low write activity, the WAL LSN won't advance properly, causing syncs to silently return 0 records. [Postgres Troubleshooting] Fix — Set up a heartbeat table:

CREATE TABLE airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL DEFAULT current_timestamp,
    text TEXT
);

Then add it to your publication:

ALTER PUBLICATION <publicationName> ADD TABLE airbyte_heartbeat;

Then configure the Debezium heartbeat query in the connector settings:

INSERT INTO airbyte_heartbeat (text) VALUES ('heartbeat')

3. WAL Resume Position Timeout The connector may be silently timing out while searching for the WAL resume position, resulting in successful syncs with 0 records. [Github Issue] Fix: Increase Initial Waiting Time in Seconds (Advanced) — try setting it to

2400

. Given that you have no errors, the most likely scenario is either the PgJDBC regression (if you're on connector >= 3.6.19) or low transaction volume on your publication tables. I'd recommend checking your connector version first. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai, yes i have low transaction volume and my postgres source connector version is 3.6.35

Given your connector version

3.6.35

and low transaction volume, the heartbeat table solution is the right fix for you. Here's exactly what to do: Set Up a Heartbeat Table Step 1: Create the heartbeat table in your source database:

CREATE TABLE airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL DEFAULT current_timestamp,
    text TEXT
);

Step 2: Add it to your Airbyte publication:

ALTER PUBLICATION <publicationName> ADD TABLE airbyte_heartbeat;

Step 3: Grant the Airbyte user write access to the table:

GRANT INSERT, SELECT ON TABLE airbyte_heartbeat TO <airbyte_user>;
GRANT USAGE, SELECT ON SEQUENCE airbyte_heartbeat_id_seq TO <airbyte_user>;

Step 4: In the Airbyte UI, go to your Postgres source connector settings → Advanced → Debezium heartbeat query and enter:

INSERT INTO airbyte_heartbeat (text) VALUES ('heartbeat')

This causes Airbyte to periodically write to the heartbeat table, which keeps the WAL LSN advancing even when your actual publication tables have little to no activity. [Postgres Troubleshooting] > Note: You do not need to sync the heartbeat table itself — it just needs to be in the publication to keep the LSN moving forward. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai, i have added the heartbeat table do i need to do anything else ?

Once you've set up the heartbeat table, here's a quick checklist to make sure everything is in order: Verification Checklist 1. Confirm the heartbeat table is in the publication:

SELECT * FROM pg_publication_tables WHERE pubname = '<publicationName>';

You should see

airbyte_heartbeat

listed. 2. Confirm the Debezium heartbeat query is configured in the Airbyte UI under your Postgres source → Advanced → Debezium heartbeat query:

INSERT INTO airbyte_heartbeat (text) VALUES ('heartbeat')

3. Confirm the Airbyte user has write access to the heartbeat table:

GRANT INSERT, SELECT ON TABLE airbyte_heartbeat TO <airbyte_user>;
GRANT USAGE, SELECT ON SEQUENCE airbyte_heartbeat_id_seq TO <airbyte_user>;

Next Steps If all the above are in place: • Trigger a manual sync and observe if incremental records start flowing. • You do not need to sync the heartbeat table itself — just having it in the publication with the heartbeat query configured is sufficient. [Postgres Troubleshooting] > Note: The first sync after this change may still return 0 records. Give it a couple of sync cycles to stabilize as the heartbeat starts advancing the LSN. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

connector version

3.6.35

has a driver-related bug where it can advance the replication slot too far, requiring a full refresh. I'd suggest

3.7.2

or

3.8.x

.