An option to truncate the final destination table instead of recreating it in the case of

Full Refresh - Overwrite

Sorry - not at this time. Especially as we are trying to build syncs with 0 data downtime, we sync data to a new table and then at the end of the sync swap the old and new tables into place. This is both faster and cheaper then running a

delete

on the existing data too.

0 data downtime

- can’t this be done in a transaction?

Where we can, yep! The swapping of the new and old final tables is done in a transaction... although some destinations (bigquery) don't offer transactional guarantees on metadata operations, of which a rename is.

instead of a swap - what about a truncate and insert? I don’t think I understand the difference, and in some destinations recreating the final table keeps us from using things like timetravel in Snowflake

It could, and in the future we might optimize this, but for the new table + rename lets us handle other use-cases as well (e.g. schema migrations and type changes) with a bit more resiliency. We have not prioritized switching to truncate+insert at this time.

🤔 - I’ll have to test that out ; in the past, we’ve had to undrop tables to get the timetravel