Hi there, just an odd behaviour I've noticed. The original sync worked well but the next syncs are showing

0 bytes | no records | no records

. This is odd since the database has for sure changed since then. Is there any documentation to troubleshoot? The syncs shows as success. Source: PostgreSQL Destination: Snowflake All tables have a PK.

Hi Alexandre, could you please include the Airbyte and connector versions as well as logs? This is not enough information to go off of at this point, unfortunately.

Of course! just a moment

Hi Alexandre, thanks again for your patience! I've looked into this further and have some questions - From the logs I see that you're using the append dedup history sync mode, is that correct? A note about this - Airbyte has to go through the whole table to check for duplicates so that is why it's taking so long. From the screencap you attached in the forum post, I see that the database is large. We need to double check that changes are actually happening in the database. I would suggest trying to do a test incremental append sync with a subset of the data to see if that works. I haven't seen any bugs come up with the Postgres source connector where it seems the issue lies in this case. Would you be able to do an incremental append sync on a small subset of your data as a test?

Regarding the cursor field: do you have an

updated_at

field in your table?

Hi @Nataly Merezhuk (Airbyte) thanks for getting back to me! I am indeed using

Incremental |  Deduped + history

on all tables. I do have an

updated_at

field (I'm using ruby on rails which implements it in its ORM). All tables have a PK (usually

id

)

I see someone has had a very similar error with a Postgres->Snowflake connection with these sync modes. https://discuss.airbyte.io/t/incremental-models-failing/2613 I think the first response is very thoughtful: They suggest going through this tutorial to see how that clustering key is calculated to debug the issue: https://airbyte.com/tutorials/incremental-data-synchronization And it looks like we also have an issue open for this: https://github.com/airbytehq/airbyte/issues/17655

thank you @Nataly Merezhuk (Airbyte)! 😄