@kapa.ai: Clearing of data streams from airbyte runs indefintely and the job history download fails and cannot view jobhistorylogs

It sounds like you’re hitting two issues at once: 1. Clear (reset) jobs never finish 2. You can’t download or view the job logs From the knowledge sources, there are two main patterns that can cause this behavior: ------------------------- 1. MinIO / log storage is full → jobs hang and logs can’t be accessed A closed GitHub issue describes “all sync jobs hanging indefinitely” that was ultimately traced to MinIO storage filling up. Deleting logs or increasing storage resolved it. [All sync jobs] Key points from that issue: • The MinIO PVC is often small by default (e.g., 500 MB) and can fill quickly with logs. • When it’s full: ◦ Jobs can hang indefinitely. ◦ Log-related operations (like downloading job logs) can fail. • Workarounds that fixed it: ◦ Delete logs inside the MinIO container, e.g.: ▪︎

/storage/airbyte-storage/job-logging/workspace

▪︎

/storage/airbyte-storage/workload/output

[All sync jobs] ◦ Or delete/recreate the MinIO storage (note: this clears all logs). [All sync jobs] ◦ Increase the storage requested for the MinIO PVC so it doesn’t fill up as quickly. [All sync jobs] ◦ Optionally reduce Temporal history retention, e.g.:

env_vars:
          TEMPORAL_HISTORY_RETENTION_IN_DAYS: 7

to limit how much history is stored. [All sync jobs] Given your symptoms (clear jobs running indefinitely + job history download failing), this disk‑space/log‑storage issue is the most likely match in the provided sources. ------------------------- 2. How to (normally) access logs for a hanging Clear job Once log storage is healthy, you should be able to: • Go to the connection’s Job history tab. • Use the three dots next to the Clear job: ◦ View logs to open the in‑app log viewer. ◦ Download logs to get a combined log file for all attempts of that job. [Browsing logs] • If needed, copy a link to job from the same menu to share with others. [Browsing logs] If the Clear job itself fails and can’t complete, the docs recommend manually dropping the destination tables so Airbyte can continue syncing correctly afterward. [Clearing data] ------------------------- Recommended next steps 1. Check MinIO / log storage usage (PVC size and free space). 2. If it’s full: ◦ Temporarily delete old logs from the MinIO paths above, or ◦ Increase the MinIO PVC size and restart the relevant pods. 3. After freeing space, retry: ◦ The Clear data operation. ◦ View / Download logs from the Job history. 4. If the Clear job still never completes even with healthy storage, you may need to: ◦ Manually drop the affected destination tables. ◦ Then run a fresh sync. [Clearing data] If you can share your deployment type (Docker vs Kubernetes) and where MinIO is running, I can help you translate this into concrete commands/steps. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.