Mateo Graciano
12/05/2025, 8:18 PMkapa.ai
12/05/2025, 8:18 PMairbyte-minio (Kubernetes/OSS Helm): - MinIO mainly stores job logs and related outputs. [[Minio cleanup](https://discuss.airbyte.io/t/3101); [S3 Minio full](https://discuss.airbyte.io/t/1857)] - You can safely delete old logs inside the MinIO volume; examples from running pods: - Older path name: /storage/airbyte-dev-logs/job-logging/workspace/ [[Minio cleanup](https://discuss.airbyte.io/t/3101)] - Newer path name: /storage/airbyte-storage/job-logging/workspace and possibly /storage/airbyte-storage/workload/output [[Syncs hanging](https://github.com/airbytehq/airbyte/issues/55812)] After deleting logs/outputs, MinIO frees disk and syncs can run again. To avoid recurrence, reduce how long Temporal keeps workflow history (this directly affects log retention in workspace): yaml env_vars: TEMPORAL_HISTORY_RETENTION_IN_DAYS: 7 Setting this env var (e.g., in your Helm values) shortens history and reduces MinIO usage. [[Syncs hanging](https://github.com/airbytehq/airbyte/issues/55812); [Log sweeping](https://discuss.airbyte.io/t/2714)] If you see an error like “Storage backend has reached its minimum free disk threshold” or XMinioStorageFull, that also indicates your MinIO disk is full and you must delete objects or increase the PVC size. [[S3 Minio full](https://discuss.airbyte.io/t/1857); [S3 + Minio issue](https://github.com/airbytehq/airbyte/issues/59197)] --- ### 2. Airbyte workspace logs on the host are too large (Docker / VM) Airbyte’s workspace (logs, job artifacts) can fill the host disk over time and cause: - Sync jobs failing - UI not loading - Commands like docker logs failing with “no space left on device” [[Syncs suddenly failing](https://discuss.airbyte.io/t/3303); [Airbyte rotation logs](https://discuss.airbyte.io/t/332)] Current behavior: - From v0.40.5+, Airbyte periodically cleans files from the workspace (default: every 30 days). [[Log sweeping](https://discuss.airbyte.io/t/2714)] - You can configure retention with: bash TEMPORAL_HISTORY_RETENTION_IN_DAYS=1 # example: keep 1 day This replaces the older MINIMUM_WORKSPACE_RETENTION_DAYS / MAXIMUM_WORKSPACE_RETENTION_DAYS / MAXIMUM_WORKSPACE_SIZE_MB mechanism, which no longer applies with the Temporal-based scheduler. [[Log sweeping](https://discuss.airbyte.io/t/2714)] If your disk is already full: - Manually delete old log folders under the Airbyte workspace directory (on many Docker installs this is something like ./data where you cloned Airbyte), or, if using MinIO as above, clear logs from MinIO. --- ### 3. Destination database temp or tmp tables Sometimes the “no space left on device” is in the destination database, not the Airbyte host: - Example: Postgres / Aurora error could not write to file "base/pgsql_tmp/..." : No space left on device appears during normalization. [[No space normalization](https://discuss.airbyte.io/t/1572); [No space root cause](https://discuss.airbyte.io/t/1572)] In that case, you must free or increase storage on the destination DB side (e.g., clean up large temp files, increase storage allocated to the DB instance). [[No space root cause](https://discuss.airbyte.io/t/1572)] Also, failed syncs may leave large _airbyte_tmp_* tables in the destination; those are not auto-deleted when a sync fails and can consume space. You can drop these _airbyte_tmp_* tables manually in the destination DB. [[S3 source tmp tables](https://discuss.airbyte.io/t/2067)] --- ### 4. Jobs database or other containers filling disk A very large jobs database (Airbyte internal DB) can also fill the disk and bring airbyte-db down, which in turn breaks the whole instance. [[Syncs suddenly failing](https://discuss.airbyte.io/t/3303)] In that case you may need to: - Increase disk allocated to the host/volume used by `airbyte-db`; and/or - Manually prune job records in the jobs DB (not directly documented here, but this is what the referenced user did by truncating jobs history). [[Syncs suddenly failing](https://discuss.airbyte.io/t/3303)] --- ### 5. Summary of concrete actions Depending on where your space is exhausted: 1. MinIO PVC full - SSH/exec into MinIO pod and delete contents in: - /storage/airbyte-storage/job-logging/workspace - /storage/airbyte-storage/workload/output (or /storage/airbyte-dev-logs/job-logging/workspace on older setups). [[Minio cleanup](https://discuss.airbyte.io/t/3101); [Syncs hanging](https://github.com/airbytehq/airbyte/issues/55812)] - Optionally increase MinIO PVC size. - Set TEMPORAL_HISTORY_RETENTION_IN_DAYS to a lower value. 2. Host workspace full (Docker/VM) - Manually remove old log/job folders under the Airbyte workspace directory. - Upgrade to ≥0.40.5 and set TEMPORAL_HISTORY_RETENTION_IN_DAYS to control retention. [[Log sweeping](https://discuss.airbyte.io/t/2714)] 3. Destination DB out of space - Increase DB storage or clean up space (including temp files or _airbyte_tmp_* tables). [[No space root cause](https://discuss.airbyte.io/t/1572); [S3 source tmp tables](https://discuss.airbyte.io/t/2067)] If you share what deployment you’re using (Docker vs Kubernetes, and whether you see MinIO-related errors), I can point to the most relevant of these steps.Mateo Graciano
12/05/2025, 8:43 PMkapa.ai
12/05/2025, 8:43 PMkapa.ai
12/05/2025, 8:44 PMdocker image prune or docker system prune are safe for Airbyte as long as you don’t remove Docker volumes, because all Airbyte configs/state are stored in volumes, not in images or running containers. [[Persistence GCP](https://discuss.airbyte.io/t/1381); [Compose up/down](https://discuss.airbyte.io/t/3836)] - A docker-compose down (or docker compose down) does not delete volumes by default, so stopping the stack and pruning unused images afterwards will not remove your connections or config. [[Compose up/down](https://discuss.airbyte.io/t/3836)] Be careful with any prune command that includes `--volumes`; the knowledge sources don’t describe that scenario, and deleting Airbyte volumes (e.g. airbyte_db, airbyte_workspace) would remove data.