https://linen.dev logo
Join Slack
Powered by
For additional context. I have 3 source tables wit...
# troubleshooting
l
For additional context. I have 3 source tables with a shared join key. After every Airbyte replication job, at least one of the three relationship tests on that join key fails, because one of the table is always missing records that the other two are not (by a few seconds) at the time of replication.
After looking through the logs, I see that each individual table stream has its own cursor for 
updated_at
on incremental syncs, and therefore for multiple tables it will pickup different 
updated_at
timestamps for which to search. And so any new records created between the updated_at of the first dependent table and downstream dependencies will get lost until the next job run.
a
Hey @Lucas Wiley, do you mind sharing a concrete example of the problem you encounter? @Chris Duong [Airbyte] is it possible to have 
created_at > _airbyte_emitted_at_
? 🤔
c
created_at
is not an airbyte column, though, so i’m not sure how your source is populating that column (it could use a date in the future compared to the 
airbyte_emitted_at
or using a different timezone?)
or from the comments, it seems like we are talking about 
created_at
and`_airbyte_emitted_at` timestamp from different tables/entities?
Open in Slack
PreviousNext
Powered by