Hell team, I'm trying to install Airbyte on EKS using helm chart. I want to use my RDS postgres db instead of default one which came through helm charts. Documentation says that I can install it. All I've to do is disable default postgres helm chart and provide external db creds in the values.yaml. But the issue is, secret doesn't created with external db creds. Can you please let me know what else I've to do? https://docs.airbyte.com/deploying-airbyte/on-kubernetes#:~:text=planned%20logging%20improvements.-,U[…]0an%20external%20DB,-After%20Issue%20%233605 https://github.com/airbytehq/airbyte/blob/f927dc501d05dc1030d130c3eb5544e5430af6c2/charts/airbyte/values.yaml#L559

@Davin Chia (Airbyte) is it possible to help here ?

Hmm I haven’t had time to explore how this works with external DBs

Thank you @Davin Chia (Airbyte) for your response. I've deployed it via Kustomize & configured external RDS postgres. But I''m facing an issue with Temporal service... It would be highly appreciated if you can help here. It keeps on restarting.

does Temporal have the right database configuration?

@Davin Chia (Airbyte) I've provided the following db URL to temporal service. Pod is up now, but If I see its logs, it is not able to connect to db

jdbc:<postgresql://cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com:5432/postgresql>

postgresql

database is created. Temporal service is not able to connect. Following are the tempral service logs

can you try and pass in the ip?

@Davin Chia (Airbyte) RDS postgres has no IP address. It has public endpoint.

your RDS is open to the internet?

No, It is not exposed as public. It is in private subnets.

all DNS names can be converted to ip - e.g.

nslookup <http://cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com|cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com>

. you can google to find out more information you can look at the the original temporal chart to see what is passed in. either the dns name or the ip should work. I’m sorry but we don’t offer infrastructure debug help - each company’s infrastructure is set up too differently for this to be practical. my general tips are to double check the env vars are given the right values and that the pod is receiving them! I’d also check that there are no networking restrictions on your RDS instance

@Davin Chia (Airbyte) I've cross checked my ENV variables, they seems to be alright. As whatever database I provide, when airbyte spins up, it creates all the tables into that datbase. But the same database is not accessible by Temporal service. Following are the env to temporal service:

DATABASE_USER=<user>
DATABASE_PASSWORD=<password>
DATABASE_HOST=<http://cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com|cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com>
DATABASE_PORT=5432
DATABASE_DB=postgresql
DATABASE_URL=jdbc:postgresql://${DATABASE_HOST}:${DATABASE_PORT/${DATABASE_DB}
DATABASE_URL=jdbc:<postgresql://cdpass-dev-postgres-instance.cxlcw2mudakk.us-east-2.rds.amazonaws.com:5432/postgresql>

@Muhammad Imtiaz if you don’t try to use an external db for temporal; are you able to run Airbyte?

@Marcos Marx (Airbyte) yes, I'm able to use external db for other components. The only issue is temporal doesn't able to communicate with external db.

@Muhammad Imtiaz Airbyte uses the default image of temporal.io so if you want to edit and use an external database for temporal you need to check their docs/configuration, check some discussions here

@Marcos Marx (Airbyte) I figured out that

temporal

looks for database named

postgresql

. It doesn't work with anyother db names. Thank you very much for all your help.