Good morning, Airbyte is great but has a major iss...
# feedback-and-requests
Good morning, Airbyte is great but has a major issue with regards to tmp tables being created and not vacuumed after failed syncs (we ingest into the postgres connector). This creates out of disk memory situations in our setup constantly. In our setup, Microsoft SQL server and amazon ads connectors in particular are triggers of this bug. I am thinking about setting up a a cron that drops tmp tables that are not referenced by a running sync. Any ideas how to find those through SQL?
Shouldn't all tmp tables be dropped after a sync ? be it failed or not
Hey can you help us with the airbyte version you are using? As Anatole says tmp tables should be deleted
I wrote a python script to cleanup _tmp tables that are older than 2 hours. Will be run with cron. Maybe it is of use for anyone...
As a workaround I add a post-hook in my dbt transformation to drop the temp tables