@kapa.ai, i am replicating data from postgres to s3 in cdc manner but when i record gets deleted in the source in destination the deleted_at column for that record doesn't get updated and its still null

Thinking completed When replicating CDC data from Postgres to an append-oriented/file-based destination like Amazon S3, here is how deletions work and why your existing rows still show

_ab_cdc_deleted_at

as `null`: 1. Airbyte emits a new record for deletes rather than updating existing files S3 is an append-only/file destination and does not perform in-place SQL updates or hard deletes on previously written JSONL/CSV/Parquet files. When a row is deleted in the Postgres source: • The original insert/update record already written to S3 is *not modified*—its

_ab_cdc_deleted_at

field will permanently remain

null

[Postgres CDC ELT; MySQL CDC]. • Airbyte captures the

DELETE

event from the WAL and writes a new, separate record to the destination. In this new record,

_ab_cdc_deleted_at

contains the deletion timestamp, and only the primary key fields are populated (other data fields are typically unset/null) [Postgres Troubleshooting; Postgres CDC ELT]. Look for a second record sharing the same primary key in your S3 output files; you should see that row containing the non-null

_ab_cdc_deleted_at

value [Postgres CDC ELT; MySQL CDC]. 2. Things to check if no deletion record appears at all If no new record containing

_ab_cdc_deleted_at

appears in S3 after a sync, check the following source requirements: • Primary Key Requirement: CDC incremental replication in Postgres is only supported for tables with primary keys [Postgres Troubleshooting; CDC Documentation]. • Replica Identity: Ensure your Postgres table has a replica identity set. Normally this is

DEFAULT

, or

FULL

if you use TOAST/large fields:ALTER TABLE your_table_name REPLICA IDENTITY DEFAULT;_(or

REPLICA IDENTITY FULL;

)_ [Cloud SQL Setup]. • Table in Publication: Confirm that the table was added to your CDC publication (

CREATE PUBLICATION ... FOR TABLE ...

). Tables not in the publication are not replicated via CDC [Cloud SQL Setup]. • Delete Method: Deletions must be made via standard

DELETE

statements. DDL operations like

TRUNCATE

will not appear in replication logs [Postgres Troubleshooting; CDC Documentation]. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.