Hi all, I have an issue where for some records we ...
# replication-troubleshooting
m
Hi all, I have an issue where for some records we have multiple days difference between
_airbyte_emitted_at
and
_airbyte_normalized_at
We run this job daily (on working days), and in most cases it works how we expect it to work, the difference between these times is at most a few minutes. Could someone clarify what could cause days of difference, as seen in the row I highlighted on the screenshot? Thank you! 😊
u
Hello Mark Suemegi, it's been a while without an update from us. Are you still having problems or did you find a solution?
m
Hi, we are now using
_airbyte_emitted_at
as the "ingestion date". I still don't understand exactly how a record emitted on the 23rd is only normalized on the 28th. What I expected that If I run the connection daily, for every day I'll get new values (the date when the connection is triggered) for both of these columns. Is it the case that for incremental models, if a record is unchanged, then we don't have a new date for
_airbyte_emitted_at
?
u
Hey Mark, For incremental models, if a record is unchanged, then the
_airbyte_emitted_at
value will be the same as the previous sync. This is because the record is not re-fetched from the source, and only new or modified records are replicated.For more information, please refer to the Incremental Append section of the Airbyte documentation. Does that answer your question?