What are some things I can do to avoid this error on postgres cdc? "Replication slot 'your_slot' has advanced beyond the source's state LSN. Confirmed flush LSN: LSN{327F1/C5582CF8}, source LSN: LSN{327F1/C46B8B60}."

Thinking completed This error means the replication slot's

confirmed_flush_lsn

has advanced past the LSN saved in Airbyte's state — a known bug in source-postgres versions >= 3.6.19. Here are your options: Immediate Workarounds 1. Downgrade to source-postgres v3.6.18 (or v3.6.15) The most reliable short-term fix. The issue was introduced in v3.6.19 due to a PgJDBC regression (version 42.7.0+) that automatically advances

confirmed_flush_lsn

even when there are no new events for the publication tables. [CDC LSN issue] 2. Upgrade to source-postgres v3.7.2+ A fix was released in v3.7.2 (and a preview build

3.7.1-preview.8ed70f5

). Users have confirmed this resolves the issue. [CDC LSN issue] 3. Set up a heartbeat table This is the most robust workaround if you can't change connector versions. It keeps the LSN advancing properly by writing small events to a dedicated table in the publication: 1. Create the heartbeat table:

CREATE TABLE public.airbyte_heartbeat (
    id SERIAL PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL DEFAULT current_timestamp,
    text TEXT
);

2. Add it to your Airbyte publication:

ALTER PUBLICATION <publicationName> ADD TABLE public.airbyte_heartbeat;

3. In the Postgres source connector settings, configure the Debezium heartbeat query field:

INSERT INTO public.airbyte_heartbeat (text) VALUES ('heartbeat');

This ensures that even when your tracked tables have low traffic, the publication sees activity and the LSN advances correctly. [WAL troubleshooting] 4. Add a high-traffic table to the publication (read-only option) If you don't want to give Airbyte write access, add a high-traffic table to the publication. You don't need to sync it — it just keeps the LSN moving:

ALTER PUBLICATION <publicationName> ADD TABLE <high_volume_table>;

Note: this may increase disk consumption. [WAL troubleshooting] 5. Change "Invalid CDC position behavior" to "Re-sync data" This avoids hard failures by triggering a full re-sync instead. However, it's not ideal for large tables since it will silently perform a full refresh whenever the CDC state is invalidated. [CDC LSN issue] Root cause summary: The bug stems from PgJDBC >= 42.7.0 automatically flushing the LSN when there are no new events for publication tables (even if the WAL has changes for tables outside the publication). This was introduced in source-postgres v3.6.19. [PgJDBC issue] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai I aleady have a heartbeat table set up though? But it still failed?

Thinking completed That's a known situation — even with a heartbeat table configured, the issue can persist in certain versions. Based on the knowledge sources, here's what's reported: Why the Heartbeat May Not Be Enough One user reported that even after correctly configuring both a Debezium heartbeat query and a pg_cron heartbeat, their very first sync still failed after several hours. [CDC LSN issue] The heartbeat workaround is most effective for preventing the issue after a successful initial sync. The deeper problem — particularly during the initial snapshot — is that

PostgresCdcCtidInitializer

calls

pg_current_wal_lsn()

multiple times and commits a higher LSN than what gets saved in the state. The heartbeat table doesn't fully protect against this race condition. [wrong LSN issue] What You Should Do The most reliable fix is to upgrade to source-postgres v3.7.2, which community contributors have confirmed resolves the issue: [CDC LSN issue] > "Airbyte released 3.7.2 which fixes this issue for me" If upgrading isn't immediately possible, the next best option is to downgrade to v3.6.18 as a temporary measure while you plan the upgrade. [CDC LSN issue] I'm sorry, but the knowledge sources don't contain enough detail to explain exactly why the heartbeat failed in your specific case beyond what's described above. If the issue persists after upgrading to v3.7.2, it may be worth opening a GitHub issue with your connector version and logs. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai We're using Postgres v3.8.5, but the fix was there on 3.7.2, right?

Thinking completed That's a reasonable assumption, but I'm not fully certain it applies to your situation. Here's what the knowledge sources do tell us: • The fix in v3.7.2 addressed the LSN advancement issue that was introduced in v3.6.19. [CDC LSN issue] • However, v3.8.0 introduced a separate Kotlin/bulk-CDK rewrite of the connector, which brought its own regressions (e.g. PostGIS nulls on CDC path). [PostGIS regression] • There is also a separate PR (#79671) specifically targeting v3.8.x that fixes two bugs causing

confirmed_flush_lsn

not to advance — a race condition in

ReplicationSlotManager.advanceLsn()

and a missing LSN fallback after the initial snapshot phase. [LSN fix PR] So the short answer is: the v3.7.2 fix did not carry over cleanly into the v3.8.x rewrite, and the LSN advancement problem resurfaced in a different form. I'm sorry, but the knowledge sources don't explicitly confirm whether PR #79671 has been released in v3.8.5 specifically. I'd recommend: 1. Checking the source-postgres changelog to see if the fix from PR #79671 is included in your version. 2. If not, upgrading to the latest available version of source-postgres. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai What are the some things v3.7.2 doesn't have v3.8 have that I would miss?

Thinking completed Based on the knowledge sources, I can only highlight a few things mentioned in the context of the v3.8.x rewrite, but I must be upfront that the knowledge sources don't contain a comprehensive feature comparison between v3.7.2 and v3.8.x. Here's what the knowledge sources do mention about what v3.8.x brought: What v3.8.x Introduced Over v3.7.2 Bulk CDK Rewrite • v3.8.0 was a significant Kotlin/bulk-CDK rewrite of the connector. [PostGIS regression] This is the foundational change underpinning most other improvements. CDC Payload Size Handling • v3.8.0-rc.12 added a fix for oversized CDC rows — raising the Jackson per-string cap and adding an actionable error message when CDC rows exceed the limit. [CDC payload fix] v3.7.2 would silently fail on these. Improved Array Type Discovery • v3.8.x improved handling of arrays of types outside the

search_path

(e.g.

s2.kind[]

), discovering them as

array<string>

rather than

string

. [array type fix] I'd strongly recommend checking the official source-postgres changelog for a complete picture, as the knowledge sources only cover a subset of changes. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.