I’m moving data from Postgres into Snowflake using...
# advice-data-transformation
n
I’m moving data from Postgres into Snowflake using CDC with the incremental deduped + history mode. I have no need for the scd tables that get created in Snowflake. Would it be safe do drop those via dbt directly after each sync or does Airbyte need the data in the scd tables from the previous sync for the next sync?
j
Do you mean you have no need for those tables... now? or are you 100% sure no time in the future you'll ever need to go look back at a previous state of data? I would consider the costs of keeping it and if they minimal, just leave them be.
n
Yes, 100% sure that I’ll never need them. I also suspect that the scd tables generate considerable computing cost as Snowflake’s automatic clustering optimizes them. Another reason why I want to delete the data in the scd tables is because we have to sometimes delete data for GDPR compliance. If I don’t have to think about cleaning up those tables then it’s one less problem to worry about.
j
Airbyte uses its own record for any necessary high water marks. You're safe to remove any tables you don't need.