@kapa.ai does airbtye connection deletes all the data during a full refresh for a stream when it starts running

Full Refresh and Data Deletion Behavior It depends on which Full Refresh variant you're using: Full Refresh | Overwrite Yes — this mode destroys all existing data in the destination table and replaces it with the newly synced data. However, for certified database/data warehouse destinations, Airbyte does this without causing data downtime by: 1. Writing new data to a temporary table (

final-table-tmp

) while the sync runs 2. Dropping the old final table and renaming the temp table into place at the end This means the previous data remains queryable until the sync completes successfully. [Full Refresh Overwrite] > ⚠️ For some destinations (like S3), the old data is deleted upon a successful sync, and Airbyte recommends provisioning a dedicated resource to avoid accidental data loss. [S3 Sync Modes] Full Refresh | Append No — this mode does not delete existing data. It appends the entire source dataset to the destination table on every sync, which can result in duplicates. [Full Refresh Append] Full Refresh | Overwrite + Deduped Similar to Overwrite — all existing data is replaced with the new sync data, but the final table is also deduplicated per primary key. This variant is recommended if avoiding duplicates (e.g., from resumed syncs) is a strong requirement. [Full Refresh Overwrite + Deduped] Summary: 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements

@kapa.ai For some destinations (like S3), the old data is deleted upon a successful sync, and Airbyte recommends provisioning a dedicated resource to avoid accidental data loss. [S3 Sync Modes] what abut gcs case?

GCS Full Refresh Behavior For the GCS destination, the documentation states: > ⚠️ Under Full Refresh Sync mode, data in the configured bucket and path will be wiped out before each sync. Airbyte recommends provisioning a dedicated GCS resource for this sync to prevent unexpected data deletion from misconfiguration. ⚠️ [GCS Configuration] This is slightly different from S3, where deletion happens after a successful sync. For GCS, the old data is wiped before the new sync begins, which means there is a higher risk of data loss if the sync fails midway. As with S3, Airbyte recommends using a dedicated GCS bucket/path for Full Refresh connections to avoid accidental data deletion. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai I am using a dedicated bucket path for gcs

That's a good practice and aligns with Airbyte's recommendation for GCS Full Refresh syncs. [GCS Configuration] By using a dedicated bucket path, you avoid the risk of accidentally wiping out unrelated data when a Full Refresh sync runs, since all data in the configured bucket path is wiped before each sync. A few additional things to keep in mind for your GCS setup: • Each stream is written to its own directory under the bucket path. • Under Full Refresh, old output files in that directory are purged before new files are created. • Under Incremental - Append, new output files are simply added, so no data is deleted. [GCS Output Schema] The knowledge sources do not contain further specific guidance beyond recommending a dedicated resource, so if you have a more specific concern (e.g., sync failures mid-wipe, data recovery), feel free to share more details. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai is there fix in a latest version of gcs destination connector, like in s3, we want to delete the data only after sucessful sync?

I'm sorry, but the knowledge sources do not contain enough information to confirm whether the GCS destination connector has a fix similar to S3's behavior of deferring deletion until after a successful sync. For context, the S3 connector introduced this improvement in version 1.0.1 (August 2024): > "OVERWRITE MODE: Deletes deferred until successful sync." [S3 Changelog] However, reviewing the GCS destination changelog, there is no equivalent fix mentioned. The latest GCS connector version is 0.4.9 (March 2025), and none of the changelog entries reference a similar deferred-deletion improvement. [GCS Changelog] To get a definitive answer, I'd recommend: • Checking the GCS connector GitHub issues/PRs directly for any open or merged work on this behavior. • Submitting a feature request if this improvement doesn't exist yet for GCS. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.