Hi, I'm trying to setup Airbyte within GKE, using an external Cloud SQL (Postgres) instance, however I'm unable to get the connection to the database working when the airbyte-bootloader pod runs. The error I receive is:

org.jooq.exception.DataAccessException: Error getting connection from data source HikariDataSource (HikariPool-2) at org.jooq_3.17.8.POSTGRES.debug(Unknown Source) ~[?:?]

The Cloud SQL instance is currently set to have a public IP address, and I have Cloud NAT enabled for the cluster, so external connectivity shouldn't be an issue. I'm deploying the helm chart with the following value overrides:

global:
  database:
    secretName: "airbyte-db-creds"
    secretValue: "DATABASE_PASSWORD"
    # the public IP of the Cloud SQL instance
    host: "39.74.12.211"
    port: "5432"

postgresql:
  enabled: false

externalDatabase:
  host: "39.74.12.211"
  user: airbyte
  password: "topsecret"
  # existingSecret: "airbyte-db-creds"
  existingSecret: ""
  # existingSecretPasswordKey: "DATABASE_PASSWORD"
  existingSecretPasswordKey: ""
  database: airbyte-db
  port: 5432
  jdbcUrl: "jdbc:<postgresql://39.74.12.211:5432/airbyte-db>"

I've setup a secret with values for the DATABASE_USER and DATABASE_PASSWORD keys. Can anyone suggest what I need to do to get this working correctly? I can successfully install Airbyte with the

postgresql

enabled setting set to true, but would prefer to use an external database.

Based on the information provided in the extracts, it seems like you're on the right track with your configuration. However, there are a few things you might want to check or adjust: 1. Database Connection URL: In your Helm chart values, the

jdbcUrl

seems to have a typo. It should be

jdbc:<postgresql://39.74.12.211:5432/airbyte-db>

instead of

jdbc:<postgresql://39.74.12.211:5432/airbyte-db>

. 2. SSL Connection: Airbyte Cloud only supports connecting to your Postgres instances with SSL or TLS encryption. If you're connecting to a Cloud SQL instance, you might need to include SSL parameters in your JDBC URL. For example, as mentioned in the [Configuring the Airbyte Database guide](https://docs.airbyte.com/operator-guides/configuring-airbyte-db), the JDBC URL could look like this:

DATABASE_URL=jdbc:<postgresql://host.docker.internal:3000/postgres?ssl=true&sslmode=require>

You might need to adjust this to match your Cloud SQL instance details and SSL settings. 3. Network Access: Ensure that your Postgres database can be accessed by Airbyte. If your database is within a VPC, you may need to allow access from the IP you're using to expose Airbyte. This was mentioned in the [Postgres destination prerequisites](https://docs.airbyte.com/integrations/destinations/postgres). 4. Database Backup: As mentioned in a [forum discussion](https://discuss.airbyte.io/t/2903), Airbyte doesn't automatically migrate data from the internal database to an external one. You'll need to manually backup and restore your data. 5. Database User and Password: Make sure the

DATABASE_USER

and

DATABASE_PASSWORD

are correctly set in your Kubernetes secret and match the credentials of your Cloud SQL instance. If you're still facing issues, it might be helpful to check the [Airbyte community discussions](https://discuss.airbyte.io) or [Slack channel](https://airbytehq.slack.com) for similar issues or further assistance.

@kapa.ai the database connection URLs in your example appear to be the same. I've tried setting the SSL connection arguments in the JDBC connection string:

jdbcUrl: "jdbc:<postgresql://39.74.12.211:5432/airbyte-db?ssl=true&sslmode=require>"

but still get the same error. The Postgres database is running in the same VPC as the GKE cluster where Airbyte is installed. The Kubernetes secret was configured with the following command:

kubectl create secret generic airbyte-db-creds --from-literal=DATABASE_USER=airbyte --from-literal=DATABASE_PASSWORD=topsecret

Is there any way to log more information on the connection failure within the bootloader pod?

Based on the information provided in the extracts, it seems like you're having trouble connecting to your Postgres database from Airbyte deployed on a Kubernetes cluster. From the [documentation](https://docs.airbyte.com/operator-guides/configuring-airbyte-db), the connection string format seems correct. However, it's important to note that the

DATABASE_URL

and

CONFIG_DATABASE_URL

should point to your specific database host, port, and database name. For example:

DATABASE_URL=jdbc:postgresql://<host>:<port>/<database>?<extra-parameters>

In your case, it would be something like:

DATABASE_URL=jdbc:<postgresql://39.74.12.211:5432/airbyte-db?ssl=true&sslmode=require>

As for logging more information on the connection failure within the bootloader pod, the extracts do not provide specific information on how to do this. However, you can generally check the logs of a Kubernetes pod using the

kubectl logs <pod-name>

command. If you're still having issues, it might be helpful to check the [Airbyte forum](https://discuss.airbyte.io) or [Slack channel](https://airbytehq.slack.com) for similar issues or further assistance.