Hello Everyone, I am deploying Airbyte via Helm on...
# ask-ai
r
Hello Everyone, I am deploying Airbyte via Helm on Azure Kubernetes. I have setup an external database, but I am wondering about Minio. Is anything other than logs are stored in Minio that should be persisted more long term? I have found mixed answers regarding this on the forum and elsewere, and I was hoping someone here could clear things up for me. I want to be able to completely delete airbyte from the cluster (or even delete the cluster itself) and re-deploy it without losing any airbyte configuraton, so it can more or less pickup where it left off again.
k
A message from kapa.ai
Airbyte uses Minio to store logs by default in a Kubernetes deployment. However, it also uses Minio for something called "storage state" [source](https://airbytehq.slack.com/archives/C021JANJ6TY/p1675159252328839). Unfortunately, there isn't enough information in the provided extracts to give you a confident answer on how to persist the "storage state" using an external storage solution like S3. For logs, you can configure Airbyte to use an external storage solution like S3 by setting the following environment variables in your Kubernetes deployment:
Copy code
S3_LOG_BUCKET=<your_s3_bucket_to_write_logs_in> AWS_ACCESS_KEY_ID=<your_s3_access_key> AWS_SECRET_ACCESS_KEY=<your_s3_secret_key> S3_MINIO_ENDPOINT=<endpoint_where_minio_is_deployed_at> S3_LOG_BUCKET_REGION= S3_PATH_STYLE_ACCESS=true
[source](https://docs.airbyte.com/deploying-airbyte/on-kubernetes#configure-logs) Regarding the Airbyte configuration, it is stored in a Postgres database. You can use an external Postgres instance to persist your Airbyte configuration and ensure that it can be restored when redeploying Airbyte. You can find more information on configuring the Airbyte database [here](https://docs.airbyte.com/operator-guides/configuring-airbyte-db). However, I don't have enough information to provide a complete solution for persisting the "storage state" outside of Minio. I recommend checking the [Airbyte documentation](https://docs.airbyte.com) and [Airbyte community forum](https://discuss.airbyte.io) for more information on this topic.
r
@kapa.ai What is the Airbyte "storage state" that is stored in Minio? Is this temporary data?
k
A message from kapa.ai
New message text here