Good morning, Airbyte is great but has a major iss...
# feedback-and-requestsj
Jens
01/27/2022, 8:29 AMGood morning, Airbyte is great but has a major issue with regards to tmp tables being created and not vacuumed after failed syncs (we ingest into the postgres connector). This creates out of disk memory situations in our setup constantly. In our setup, Microsoft SQL server and amazon ads connectors in particular are triggers of this bug. I am thinking about setting up a a cron that drops tmp tables that are not referenced by a running sync. Any ideas how to find those through SQL?
Jens
01/27/2022, 8:38 AMShouldn't all tmp tables be dropped after a sync ? be it failed or not
Jens
01/27/2022, 9:15 AMHey can you help us with the airbyte version you are using? As Anatole says tmp tables should be deleted
Jens
01/27/2022, 9:50 AM0.35.3-alpha
Jens
01/27/2022, 9:53 AMI wrote a python script to cleanup _tmp tables that are older than 2 hours. Will be run with cron. Maybe it is of use for anyone...
p
Ping-Lin Chang
01/28/2022, 1:21 AMHere is the issue: https://github.com/airbytehq/airbyte/issues/7011
Ping-Lin Chang
01/28/2022, 9:53 AMAs a workaround I add a post-hook in my dbt transformation to drop the temp tables
3 Views