Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Has anyone successfully installed Airbyte on K8s on AWS?

HI guys, we're also dealing with k8s deployment. <@U03JYN3N0HJ> did you managed to use external postgres? Which helm chart version did you use? :slightly_smiling_face:

<@U01MMSDJGC9> turned this thread into Zendesk ticket <https://airbyte7538.zendesk.com/agent/tickets/2608 | 2608> to ensure timely resolution!

Yes, we have a connection that is Postgres to Postgres, and we’re using the latest version of the chart `0.40.17`

For us the whole deployment gets stuck on `airbyte-db` pod whenever we add external database configuration in `values.yaml` . Current deployment is done with kustomize and we'd like to migrate to using helm. Since we're not that familiar with airbyte, it's taking a while. Would you be willing to share how did you configure the whole thing? Without any obvious hosts/secrets/etc.? :slightly_smiling_face:

Ah, I was confused by your question.  I do not have an external postgres DB setup as the application db, sorry!

I will likely set one up soon, but sorry I can’t help now. Is there any output from the pod not starting?

wrong credentials, but as far as I understand, airbyte-db pod shouldn't really be needed if we're using external postgres anyways. We're trying to get the official charts to work, but will probably need to write our own for now, since it looks like the officials ones are in active development and not very stable at the moment.

But if you manage to get them to work, please, do share :pray:

Great to see everyone's input here! Emilija, I saw a related issue on the forums - perhaps there could be a firewall/permissions issue with your setup?
<https://discuss.airbyte.io/t/error-deploying-airbyte-with-an-external-postgres-database-aws-rds/2179>

We can connect to external postgres via psql from inside the pod, so it shouldn't be a firewall issue, unfortunately. Current deployment mode is done via kustomize and airbyte successfully connects to external pg, so we're asuming this is related to our new setup using helm charts. Thank you, though.

Airbyte db should me disabled if you’re using external db config, to do so make sure you set db.enabled to false

Then the helpers.tpl should pick the right configuration for db connection

this is what i am stuck at "waiting for a volume to be created, either by external provisioner or manually created by system administrator

Are you trying to deploy to eks on fargate or ec2?

If possible do create an issue on airbyte github repo and include the following:
Version of chart deployed
Cluster setup
Values.yaml that you’ve provided(without sensitive data)