How to purge minio log files on abctl oss

To purge or clean up Minio log files in an Airbyte deployment on Kubernetes using

abctl

, you can manually delete the files by executing a command in the Minio pod. Here are the steps: 1. First, you need to get the name of the Minio pod. You can do this by running the following command:

bash kubectl get pods -n airbyte-abctl

This will list all the pods running in the

airbyte-abctl

namespace. Look for the pod that starts with

airbyte-abctl-airbyte-minio-0

. 2. Once you have the name of the Minio pod, you can execute a command in the pod to delete the log files. Run the following command:

bash kubectl exec -it -n airbyte-abctl airbyte-abctl-airbyte-minio-0 -- /bin/bash

This will open a bash shell in the Minio pod. 3. Now, navigate to the directory where the log files are stored. According to the information provided, the logs are stored in the

/storage/airbyte-storage/job-logging

directory. So, run the following command:

bash cd /storage/airbyte-storage/job-logging

4. Finally, you can delete the log files. If you want to delete all the log files, you can use the

rm

command like this:

bash rm *

This will delete all files in the current directory. Please note that this is a manual process and you need to be careful while deleting files to avoid removing important data. Currently, there doesn't seem to be an automatic way to clean up Minio storage space in Airbyte. Sources: - [Github Issue (open): [platform] airbyte-minio-0 pod restarts because of storage space is full](https://github.com/airbytehq/airbyte/issues/44556) - [Forum Post: How to clean up airbyte-minio and what's stored in minio?](https://discuss.airbyte.io/t/3101) - [Forum Post: S3 Minio Storage Full](https://discuss.airbyte.io/t/1857) - [Documentation > Contributing-to-airbyte > Developing Locally > Prerequisites > Using abctl for Airbyte development Part 2](https://docs.airbyte.com/contributing-to-airbyte/developing-locally#using-abctl-for-airbyte-development)

@scott run this command when you are in the pod :

mc rm --recursive --older-than 7d --force /storage/airbyte-storage/job-logging/workspace
mc rm --recursive --older-than 7d --force /storage/airbyte-storage/workload/output
mc rm --recursive --older-than 7d --force /storage/airbyte-storage/app-logging/workspace