I am deploying Airbyte using helm chart, while starting a new sync job the created pod ends in ERROR state, and when I check pod's log I found this

Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17F81E35114B9071; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null)

the sync job is from postgres db to google bigquery with loading data first in GCS I don't use AWS S3, does this error relates to the minio service

Yes, the error message you're seeing is related to the MinIO service. MinIO is an open-source high-performance object storage service that is API compatible with Amazon S3. Airbyte uses MinIO for storing logs and state information. The error message

XMinioStorageFull

indicates that the storage backend (in this case, MinIO) has reached its minimum free drive threshold. This means that MinIO has run out of space to store new data. To resolve this issue, you can manually delete some objects from MinIO to free up space. One user in a [forum post](https://discuss.airbyte.io/t/1857) mentioned that they were able to resolve this issue by SSHing into the MinIO pods and deleting all logs under the storage path. If you're frequently running into storage issues, you might want to consider increasing the storage allocated to MinIO or implementing a log rotation policy to automatically delete old logs. Please note that these actions should be performed with caution, as deleting objects from MinIO could result in data loss if not done correctly. Always ensure you have a backup of important data before performing these actions.

@kapa.ai which data loss you mean here

Please note that these actions should be performed with caution, as deleting objects from MinIO could result in data loss if not done correctly. Always ensure you have a backup of important data before performing these actions.

The data loss referred to here is related to the deletion of objects from MinIO, a high-performance, AWS S3 compatible object storage system. In the context of Airbyte, MinIO is used to store logs and other data. If objects are deleted from MinIO without proper caution, it could result in the loss of important data such as logs or other information stored in it. For instance, in one of the forum posts, a user mentioned that MinIO mainly stores the logs and deleting the MinIO storage resolved their issue. They executed a command to delete the files and folders inside

/storage/airbyte-dev-logs/job-logging/workspace/

in the MinIO pod. If these logs are important and no backup is available, deleting them could result in data loss.

bash kubectl exec into the minio pod and delete the files and folders inside `/storage/airbyte-dev-logs/job-logging/workspace/`.

(Source: [Forum Post: How to clean up airbyte-minio and what's stored in minio?](https://discuss.airbyte.io/t/3101)) Therefore, it's always recommended to ensure you have a backup of important data before performing these actions.