Based on the knowledge sources, deduplication should
not be used in the following scenarios: 1.
When you don't have a primary key: According to the [incremental data synchronization tutorial](
https://airbyte.com/tutorials/incremental-data-synchronization#step-2), "because [Incremental Sync - Append](
https://docs.airbyte.com/understanding-airbyte/connections/incremental-append) sync mode does not do any deduplication, it does not require a primary key to be defined." Conversely, deduplication modes require a primary key to identify unique records. 2.
When you want to preserve all historical versions of records: The [Incremental Sync - Append + Deduped documentation](
https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes/incremental-append-deduped) explains that in deduped mode, "data in the final table will be unique per primary key" and keeps "only the latest de-duplicated data row." If you need to maintain multiple versions of the same record over time, you should use Incremental Append mode instead, which "would just add a new row with the updated data" rather than combining rows. 3.
When you need to track all intermediate states: As mentioned in the [tutorial](
https://airbyte.com/tutorials/incremental-data-synchronization#step-3), "If a record is modified multiple times between synchronization iterations, only the state of the record at the time of synchronization will be replicated to the destination. Other intermediate states will not be recorded." If tracking all intermediate changes is important, deduplication may not be appropriate. In these cases, you would typically use
Incremental Sync - Append mode instead, which preserves all versions of records without deduplication.