@kapa.ai For a sync to an azure blob storage. Is this scenario technically possible: I start a incremental append sync, to a custom defined namespace (blob folder). Then after a full refresh, I let the incrementsl syncs happen ata different folder , because I change the custom namespace format, and the fresh data will be placed there, without affecting aany other files in that folder that were before

Thinking completed Yes, this scenario is technically possible based on how the Azure Blob Storage destination works: 1. Namespace determines the folder path: The Azure Blob Storage destination uses the stream namespace as part of the output path structure, writing files according to the pattern

<stream_namespace>/<stream_name>/yyyy_mm_dd_<unix_epoch>_<part_number>.<file_extension>

[Azure Blob Storage; Azure Blob Storage]. 2. Changing the custom namespace: When you update the connection's destination namespace to a new custom namespace, subsequent syncs will begin directing their output to the new path prefix [What is a Namespace?]. 3. Incremental Append writes new files: In Incremental - Append mode, Airbyte simply writes new incremental records into new files/blobs and does not purge or overwrite existing files in the folder [Supported sync modes; Forum Post: S3 Destination guide says that incase of full refresh sync mode, the old files will be purged but its not happening]. Any pre-existing files in the newly targeted folder will remain unaffected. (Note: If you run a Full Refresh - Overwrite instead of Append, overwrite modes are designed to replace previous sync files, whereas append modes leave existing files untouched [Supported sync modes; Forum Post: S3 Destination guide says that incase of full refresh sync mode, the old files will be purged but its not happening]). 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

OK, If I do resync, or clear data, that deleted the files as well right?

Thinking completed If you trigger a Clear (formerly "Reset") or resync after changing the namespace format, the behavior for both the new and old folders is as follows: 1. In the New Folder (Current Namespace) • When you initiate a Clear, Airbyte attempts to delete the destination files/records associated with the streams configured in that connection [Clearing your data]. • Under normal clear or full refresh overwrite behavior, Airbyte attempts to identify and delete files matching the stream's path pattern (e.g., using batch delete calls) [Github Issue: Full refresh files are not being deleted; AzureBlobStorageClient.kt]. • Once the Clear succeeds, initiating a new sync backfills all historical data into this new namespace path from scratch [Clearing your data]. 2. In the Old Folder (Previous Namespace) • The files in your old folder will not be touched. • According to Airbyte documentation, if you have orphaned tables or files that are no longer being synced to because the

Destination Namespace

configuration was changed, Airbyte will not clean them up for you; they must be managed or deleted manually [Clearing your data]. 3. Pre-existing Unrelated Files in the New Folder The knowledge sources do not explicitly detail whether Airbyte would accidentally delete pre-existing, non-Airbyte files if they already existed in that new folder path. However, Airbyte tracks objects based on stream naming conventions and regex patterns rather than wiping an entire bucket or unmanaged directory blindly [Github Issue: Full refresh files are not being deleted; Forum Post: Clearing storage area in destination increasingly slow on long-lived airbyte instances]. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.