Hi team, Airbyte is amazing and it keeps on improving, I’m using it for a very long time and I’m very satisfied. Recently I found out there is a pretty BIG BUG on Airbyte when using “Incremental” syncs from Snowflake. It happened more than once, and for some reason I get duplicates (airbyte gets rows it already brought by the same cursor). Here is a link to full bug description: https://airbytehq.slack.com/archives/C021JANJ6TY/p1708941814072109 I can’t solve it on my own, how should I alert you guys about this bug? Thanks!

Hey @Harel Oshri, thanks for sharing this feedback! This is known behavior about our incremental append sync mode. We opted for this approach because we wanted to ensure that you received all your data, even at the expense of potential duplicates.

When replicating data incrementally, Airbyte provides an at-least-once delivery guarantee. This means that it is acceptable for sources to re-send some data when ran incrementally. (from our Docs)

One way to remove those duplicates would be to use our

Incremental Sync - Append + Deduped

sync mode. If you prefer not to do so, you can also dedupe downstream using your primary key.