< kapa ai> i am setting up new connection which from postgre Airbyte #replication-ask-ai

<@U04S8PLJWMT>, i am setting up new connection whi...

Syed Hamza Raza Kazmi

03/17/2026, 6:40 AM

@kapa.ai, i am setting up new connection which from postgres to s3 and it is cdc replication incremental append why i am getting this error on new connection

Failure in source: Saved offset is before replication slot's confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future. See <https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs> for more details.

kapa.ai

03/17/2026, 6:40 AM

This error on a brand new connection is a known bug in the Postgres source connector. Here's what's happening and how to fix it: Root Cause The error occurs even on first/second syncs due to a regression introduced in connector version 3.6.19+. Specifically: • A regression in PgJDBC 42.7.0+ causes it to automatically advance

confirmed_flush_lsn

even when there are no new events for the publication tables. • This causes the saved state LSN to fall behind the

confirmed_flush_lsn

, triggering the error. [CDC LSN issue] Solutions Option 1: Downgrade the connector (most reliable immediate fix) Downgrade to connector version 3.6.18 or 3.6.15, which do not have this bug. Multiple users have confirmed this resolves the issue. [CDC LSN issue] Option 2: Upgrade to 3.7.2+ A fix was released in connector version 3.7.2. If you can upgrade, this should resolve the issue. [CDC LSN issue] Option 3: Set up a heartbeat table (workaround) This keeps the LSN advancing and prevents the mismatch. Run the following on your Postgres database:

Copy code

CREATE TABLE public.airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp timestamptz DEFAULT CURRENT_TIMESTAMP NOT NULL,
    text TEXT
);

-- Grant permissions to your Airbyte user
GRANT INSERT, SELECT, UPDATE, DELETE ON TABLE public.airbyte_heartbeat TO <airbyte_user>;
GRANT USAGE, SELECT ON SEQUENCE public.airbyte_heartbeat_id_seq TO <airbyte_user>;

-- Add to your publication
ALTER PUBLICATION <your_publication_name> ADD TABLE public.airbyte_heartbeat;

Then in the Airbyte Postgres source connector UI, under Advanced → Debezium heartbeat query, enter:

Copy code

INSERT INTO public.airbyte_heartbeat (text) VALUES ('heartbeat');

[WAL troubleshooting; CDC LSN issue] Option 4: Change "Invalid CDC position behavior" In the connector's advanced settings, change Invalid CDC position behavior from

Fail sync

Re-sync data

. This will trigger a full refresh initially but may allow subsequent incremental syncs to work. Note: the error may recur. [CDC LSN issue] Recommended approach: Use Option 1 or 2 (pin to a working connector version) combined with Option 3 (heartbeat table) for the most stable setup. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

Syed Hamza Raza Kazmi

03/17/2026, 7:33 AM

@kapa.ai, i have used option 1 and option 2 but i am still facing the same issue

kapa.ai

03/17/2026, 7:34 AM

Since Options 1 and 2 (version pinning) haven't resolved the issue for you, here are the remaining approaches supported by the knowledge sources: Next Steps to Try 1. Set Up a Heartbeat Table (Strongly Recommended) This is the most commonly cited working workaround for persistent cases. Run this on your Postgres database:

Copy code

CREATE TABLE public.airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp timestamptz DEFAULT CURRENT_TIMESTAMP NOT NULL,
    text TEXT
);

GRANT INSERT, SELECT, UPDATE, DELETE ON TABLE public.airbyte_heartbeat TO <airbyte_user>;
GRANT USAGE, SELECT ON SEQUENCE public.airbyte_heartbeat_id_seq TO <airbyte_user>;

ALTER PUBLICATION <your_publication_name> ADD TABLE public.airbyte_heartbeat;

Then in the Airbyte Postgres source UI under Advanced → Debezium heartbeat query, enter:

Copy code

INSERT INTO public.airbyte_heartbeat (text) VALUES ('heartbeat');

[WAL troubleshooting; CDC LSN issue] 2. Manually Update the Connection State One user reported that manually updating the connection state to match the current

confirmed_flush_lsn

allowed the next sync to work. You can get the current value with:

Copy code

SELECT confirmed_flush_lsn FROM pg_replication_slots WHERE slot_name = 'airbyte_slot';

Note: this user found the issue returned on subsequent syncs without the heartbeat fix in place. [CDC LSN issue] 3. Add a High-Volume Table to the Publication (Read-Only Fix) If you have any table with high transaction volume, adding it to the Airbyte publication can help advance the LSN without requiring write access:

Copy code

ALTER PUBLICATION <publicationName> ADD TABLE <high_volume_table>;

[WAL troubleshooting] 4. Change "Invalid CDC Position Behavior" to
Re-sync data
As a last resort, change this setting in the connector's advanced options. It will trigger a full refresh but may restore incremental behavior afterward. [CDC LSN issue] ------------------------- The knowledge sources do not provide further debugging steps beyond these options. If none of these work, I'd recommend opening a GitHub issue directly at the Airbyte repository with your connector version, Airbyte version, and relevant logs, as this appears to be an ongoing bug being actively investigated by the Airbyte team. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

2 Views

Open in Slack

Previous Next