I'm seeing that confirmed_flush_lsn only updates a...
# replication-ask-ai
h
I'm seeing that confirmed_flush_lsn only updates after sync completes (successful sync or not), but the sync seems to be running indefinitely. Does this imply that the replication slot will continue stacking WAL records and only flush when the sync completes, or does the flushing occur while syncing but the current_flush_lsn value doesn't update until sync ends?
Copy code
source > WARN i.d.c.p.PostgresConnectorConfig(validateFlushLsnSource):1070 Property 'flush.lsn.source' is set to 'false', the LSN will not be flushed to the database source and WAL logs will not be cleared. User is expected to handle this outside Debezium.
k
Based on the information provided in the extracts, the
confirmed_flush_lsn
is indeed updated after a sync completes. This is part of the Change Data Capture (CDC) mode in Postgres where the source first runs a full refresh sync to read the snapshot of all the existing data, and all subsequent runs will only be incremental syncs reading from the write-ahead logs (WAL) [source](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting). However, if the sync is running indefinitely, it could be due to a few reasons. One possibility is that the Postgres connector is continuously reading from the WAL and acknowledging it to Postgres, but the destination connector fails to consume the data. In this case, the Postgres connector will try to read the same WAL again, which may have been removed by Postgres, since the WAL record is already acknowledged [source](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting). Another possibility is that the Postgres connector is waiting for the default initial wait time of 5 minutes (300 seconds) to start processing the data in the CDC mode. If the connector does not have enough time to create the initial snapshot or read through the change logs, you may need to increase the waiting time [source](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting). As for the warning message you provided, it suggests that the 'flush.lsn.source' property is set to 'false', which means the LSN will not be flushed to the database source and WAL logs will not be cleared. This implies that the user is expected to handle this outside Debezium. If you're experiencing issues with disk usage due to the stacking of WAL records, you might want to consider setting frequent syncs for CDC to ensure that the data doesn't fill up your disk space. If you stop syncing a CDC-configured Postgres instance with Airbyte, you should delete the replication slot to prevent it from filling up your disk space [source](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting).