@kapa.ai with incremental sync, how does airbyte know where to start from? For example is during sync the destination has interruption and got down in the middle of sync, then get bak up again after few minutes. How does airbyte handle this to avoid data lose?

Airbyte uses a concept called "state" or "cursor" to keep track of where an incremental sync should start from. This allows Airbyte to resume syncs from where they left off, even if there are interruptions. Here's how Airbyte handles this situation: 1. Checkpointing: Airbyte uses a checkpointing mechanism to save the progress of a sync. During the sync, Airbyte periodically emits state messages that contain information about the current progress [Airbyte Protocol](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol#state--checkpointing). 2. Cursor fields: For incremental syncs, Airbyte typically uses a cursor field (like a timestamp or ID) to determine which data has already been synced [Incremental Syncs](https://docs.airbyte.com/connector-development/config-based/understanding-the-yaml-file/incremental-syncs). 3. State persistence: The cursor/state information is stored in the Airbyte database. If a sync is interrupted, the next sync will use this stored state to resume from where it left off [Self-hosted Airbyte resiliency](https://discuss.airbyte.io/t/840). 4. Resumability: Airbyte is designed to be resumable. If a sync fails or is interrupted, Airbyte will automatically attempt to resume the sync from the last known good state [Resumability](https://docs.airbyte.com/using-airbyte/core-concepts/#resumability). 5. At-least-once delivery: It's important to note that Airbyte follows an at-least-once delivery model. This means that in case of failures or retries, some records might be delivered more than once to ensure no data is lost [Resumability & Resumable Full Refresh](https://docs.airbyte.com/understanding-airbyte/resumability#resumability). In the scenario you described, where the destination goes down in the middle of a sync and comes back up after a few minutes, Airbyte would handle it as follows: 1. When the sync is interrupted, Airbyte will have saved the last known good state. 2. When the destination is back up and the sync is retried, Airbyte will use this saved state to resume the sync from where it left off. 3. This ensures that data is not lost, although there might be some duplicate records due to the at-least-once delivery model. To handle potential duplicates, you can use sync modes that include deduplication, such as the Incremental Dedupe mode, if supported by your destination [Failed attempts leading to duplicates](https://discuss.airbyte.io/t/744).