Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

<@U04S8PLJWMT> I’m unable to download the logs (seeing `jobHistory.logs.logDownloadFailed`) or even view the logs when the sync volume is slightly higher.
Could you please help me understand why this is happening and suggest how I can reliably access logs during high-volume syncs?

The `jobHistory.logs.logDownloadFailed` error and inability to view logs during high-volume syncs is a known issue with a few common root causes:
*Common Causes*
*1. MinIO Storage Running Out of Space*
The most frequently reported cause is that the MinIO pod (which stores logs) fills up. By default, MinIO has only a *500MB PVC*, which can fill up quickly with frequent or large sync jobs. When storage is full, logs cannot be written or retrieved. [<https://github.com/airbytehq/airbyte/issues/55812|sync jobs hanging>]
*Fix:* Delete old logs from the MinIO container, specifically from:
• `/storage/airbyte-storage/job-logging/workspace`
• `/storage/airbyte-storage/workload/output`
You should also increase the PVC size for MinIO on your cluster.
*2. Temporal History Retention Too Long*
The default Temporal history retention is *30 days*, which contributes to log storage bloat over time. [<https://github.com/airbytehq/airbyte/issues/55812|sync jobs hanging>]
*Fix:* Reduce the retention period by adding this to your `values.yml`:
```env_vars:
  TEMPORAL_HISTORY_RETENTION_IN_DAYS: 7
```
*3. DEBUG Logs Enabled at Scale*
Running `DEBUG` logs during high-volume syncs can overload the `server` pod, preventing normal operation of the deployment, including log access. [<https://docs.airbyte.com/platform/enterprise-setup/scaling-airbyte#debug-logs|Scaling Airbyte>]
*Fix:* Turn off `DEBUG` logs for any non-testing use of Self-Managed Airbyte.
*4. Log Server Running Out of Space*
If you're running a custom log server, it may simply be running out of disk space. One user resolved intermittent log unavailability by increasing their log server's storage capacity. [<https://discuss.airbyte.io/t/551|Postgres to Snowflake>]
*How to Access Logs Reliably*
• Navigate to the connection's *Job History* tab and use the three-dot menu to *View logs* or *Download logs*. [<https://docs.airbyte.com/platform/operator-guides/browsing-output-logs|Browsing logs>]
• If a sync spans multiple attempts, downloading the logs will union all attempts into one file.
• To search across all attempts, download the logs locally rather than using the in-app viewer.
In summary, the most likely culprit for your issue is *MinIO storage being full*. I'd recommend checking and expanding its PVC size and clearing old log files as a first step. :thumbsup: 0 button :thumbsdown: 0 button :information_source: *Tip:* Mention *@kapa.ai* in the thread for follow-ups or *DM the app* for *1-to-1* help.