Hi, A couple questions for which I didn't find any answer here or in the doc : • In Deduped mode, if I delete raw and scd tables, will the following syncs fail or be any slower ? And what about tmp tables ? • When I have a long sync going on for several hours, I can see in the logs that the data from all tables is extracted before it starts writing in the destination. Does it mean that this data is temporarily stored in memory wherever Airbyte is hosted ? It sounds odd as I would expect airbyte to write it in the destination as soon as it's loaded, maybe in raw or tmp tables, to save resources. But I can see that these tables remain empty until the whole extraction phase is over Thanks

Hey Anatole 1. These tables are part of the process during the normalisation and they will be deleted once the normalisation is done. Also it's suggested we don't delete them during the process as they are part of it. 2. We have some information here https://github.com/airbytehq/airbyte/issues/3439 and bit info here also https://github.com/airbytehq/airbyte/issues/3439

Hi @Harshith (Airbyte), Thank you for your answer. Regarding 1., I don't think SCD tables are deleted as they are meant to keep every version of records. But I would like to know whether it would slow the process or make it fail to delete them in order to save storage space.

Sorry I meant temp are deleted

If you delete raw ideally nothing will break as the state is stored in the database.

By raw, you mean SCD tables as well ? Thanks

In Deduped mode, if I delete raw and scd tables, will the following syncs fail or be any slower ?

No, it won’t fail, it should work and recreate the tables. When the scd is deleted, it will rebuild the whole history from what is in the raw table, so yes it might be slower. If you also delete raw tables, then you are resetting the history to 0 by doing so. If that results into having less data than before in the raw tables, then it’d be faster to rebuild the scd tables… 😅