when using cdc with the postgres connector, every ...
# replication-ask-ai
y
when using cdc with the postgres connector, every time tehre is an update to a field, the airbyte exctracted at field gets updated and get a new timstamp, correct?
✅ 1
k
Not exactly. The
_airbyte_extracted_at
(also referred to as
_airbyte_emitted_at
in older versions) reflects when Airbyte emitted/extracted the record during a sync, not when the field was updated in the source. Here's what actually happens with CDC and updates: • When a record is updated in the source, Airbyte captures that change event from the WAL (write-ahead log) and emits a new record to the destination during the next sync. [CDC Docs] • The new emitted record will have a new
_airbyte_extracted_at
timestamp
, but only because it is a newly emitted record — not because the field itself tracks source-side update times. • The CDC-specific metadata field
_ab_cdc_updated_at
is what captures the timestamp of the actual database transaction that caused the change (INSERT/UPDATE/DELETE). [CDC Tutorial] To illustrate: when a record with
id=1
was updated twice in the tutorial example, two new raw records appeared in the destination, each with the same
_airbyte_emitted_at
(since they were captured in the same sync), but with different
_ab_cdc_updated_at
values reflecting the actual transaction times. [CDC Tutorial] In summary: •
_ab_cdc_updated_at
→ timestamp of the source database change •
_airbyte_extracted_at
/
_airbyte_emitted_at
→ timestamp of when Airbyte synced/emitted the record 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
y
@kapa.ai Okay, but my question is when ther eis a change in a record, cdc would capture it and have an updated record with a newer
_airbyte_extracted_at
k
Yes, that is correct. When CDC captures a change to a record, the newly emitted record will have a newer
_airbyte_extracted_at
(or
_airbyte_emitted_at
) timestamp compared to the original record, since it is emitted during a later sync. You can see this clearly in the tutorial example. After updating the record with
id=1
, the raw table shows the new entries with a later
_airbyte_emitted_at
timestamp: • Original record (id=1, name="A1 CDCSyn"):
_airbyte_emitted_at = 2022-09-26 18:07:07.009+00
• Updated record (id=1, name="D1 CDCSyn"):
_airbyte_emitted_at = 2022-09-26 18:19:31.857+00
[CDC Tutorial] So yes — each time a change is captured and synced, the resulting record in the destination will carry a newer
_airbyte_extracted_at
timestamp reflecting when that sync ran. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.