https://linen.dev logo
Join Slack
Powered by
Hello! I’m evaluating Airbyte and running into an ...
# replication-troubleshooting
s
Hello! I’m evaluating Airbyte and running into an issue where 
Incremental | deduped + history
is ‘succeeding’ but finding “no records” on an initial sync from an Aurora Postgres database into Snowflake. I was able to do an initial sync using the same source/destination and same replication settings for 
Full refresh
and 
Incremental | append
. My cursor is an updated_at timestamp column and the table I’m testing has a primary key. Any initial ideas on why the ‘deduped + history’ mode might not be emitting any records?
✍️ 1
✅ 1
m
Are you using the same connection and only changing the sync mode?
s
I created 3 separate connections. They each have their own stream prefix.
append works, dedupe does not write any records.
m
Can you share the sync log for dedup?
s
1bb1f3cc_f392_4709_88f0_b2cb3dd277c0_logs_17_txt (1).txt
1bb1f3cc_f392_4709_88f0_b2cb3dd277c0_logs_17_txt (1).txt
2022-10-10 181451 [44msource[0m > State Report: stream name: AirbyteStreamNameNamespacePair{name=‘users’, namespace=‘public’}, original cursor field: updated_at, original cursor value 2022-08-26T150359.057443, cursor field: updated_at, new cursor value: 2022-08-26T150359.057443
Seems maybe relevant? Is it trying to find the records between the same two timestamps?
For append, that message is:
2022-10-10 180258 [44msource[0m > State Report: stream name: AirbyteStreamNameNamespacePair{name=‘users’, namespace=‘public’}, original cursor field: null, original cursor value null, cursor field: updated_at, new cursor value: 2020-06-17T122309.156561
m
Yep, can you reset the incremental + dedup and sync again?
s
Sorry for the delay here - just wanted to say that the reset fixed the state of things! Thank you. 🙂
6 Views
Open in Slack
PreviousNext
Powered by