For connections set to Incremental Sync - Append, and the cursor set to updated_at, how does airbyte find the delta? Is it comparing the content of each row via a hash?

For connections set to Incremental Sync - Append with the cursor set to

updated_at

, Airbyte finds the delta by using the cursor field to identify new or updated records since the last sync. It does not compare the content of each row via a hash. Instead, it uses a SQL query to fetch records where the cursor field value is greater than or equal to the last recorded cursor value from the previous sync. Here is an example of how the SQL query might look:

sql SELECT * FROM table WHERE cursor_field >= 'last_sync_max_cursor_field_value'

This query ensures that only the records with a

cursor_field

value (in this case,

updated_at

) greater than or equal to the last recorded value are fetched, effectively capturing the delta of new or updated records. For more details, you can refer to the [Incremental Sync - Append documentation](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append#known-limitations). ### Key Points: - Cursor Field: The field used to track changes (e.g.,

updated_at

). - Delta Identification: Uses a SQL query to fetch records with a cursor field value greater than or equal to the last recorded value. - No Hash Comparison: The method does not involve comparing row content via a hash. For further reading, you can check the [Incremental Sync - Append Overview](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append) and [Incremental Sync - Append Rules](https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/incremental-append#rules).

right thanks for the reminder 🙂