How do historical re-syncs affect the airbyte raw ...
# ask-ai
w
How do historical re-syncs affect the airbyte raw tables? (not the final tables)
k
When Airbyte performs a re-sync, it affects the raw tables in the following ways: 1. Schema Migrations: Airbyte may drop and re-create the final table produced by a sync, sometimes referred to as a "soft reset". This is done to strictly control the column order and data types of each column. Many destinations lack the ability to control column order or alter one data type to another. Re-creating the table allows Airbyte to manage these aspects. 2. Transactional Full Refresh: To keep your final tables consistently useful, when a refresh or reset occurs, Airbyte will erase the raw tables, and then build a new temporary final table first. Airbyte attempts to do an atomic swap of old and temporary final tables, usually via a rename at the last second. This is done to avoid a period where the final table is empty, which could cause downstream issues. 3. Incremental Sync - Deduped History: If an Airbyte user has selected the Incremental Sync - Deduped History sync mode, then the data must be deduplicated so that a single entry in the source table only results in a single corresponding entry in the final normalized deduplicated destination table, even though multiple versions corresponding to that record may appear in the raw and historic data tables. Please note that additional permissions, constraints, views, or other rules you apply to the final table outside of Airbyte could be lost during a sync. It's recommended not to alter the final tables (e.g., adding constraints) as it may cause issues with the sync. Sources: - [Airbyte Core Concepts](https://docs.airbyte.com/using-airbyte/core-concepts/typing-deduping#on-final-table-re-creation) - [Incremental Data Synchronization](https://airbyte.com/tutorials/incremental-data-synchronization#step-2)