Chanakya Pendem
12/18/2025, 8:18 AMkapa.ai
12/18/2025, 8:18 AMsql SELECT * FROM table WHERE cursor_field >= last_sync_max_cursor_value Note the >=. This is intentional to provide at-least-once delivery, meaning a record with the same cursor value as the last sync can be sent again. [[Incremental append](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes/incremental-append#inclusive-cursors)] Because of this, you can see: - First sync: 3 records (cursor max = X) - Second sync: the same 3 records again (cursor = X, and cursor >= X is true) You “face” this in one of two ways: 1. Handle duplicates downstream - Use a primary key and deduplicate in your warehouse/lake (e.g., using DISTINCT, window functions, or SCD tables). - Many destinations/normalization flows are designed assuming at-least-once delivery and will remove duplicates based on primary key. [[Freshdesk forum](https://discuss.airbyte.io/t/2828); [Incremental append](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes/incremental-append#inclusive-cursors)] 2. Adjust your custom source implementation - If you control the source code, you can implement a more precise cursor (e.g., higher‑granularity timestamp or composite cursor) so that subsequent syncs don’t re-emit unchanged rows. - However, you must still respect the at-least-once guarantee; Airbyte docs explicitly allow sources to resend data when the cursor is ambiguous. [[Incremental append](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes/incremental-append#inclusive-cursors)] If you need true “exactly-once” behavior at the destination, it must be achieved via deduplication logic there; the core incremental-append protocol itself is at-least-once.