Hi, I'm trying to setup Airbyte within a GKE cluster, connecting to an external Cloud SQL database via the Cloud SQL proxy. It appears the recommended way to configure the proxy is as a sidecar within the relevant pods, and that Airbyte supports this via the

extraContainers

attribute for each pod definition in the Helm chart's values.yaml file. Can anyone suggest which pods (e.g. bootloader, db, webapp etc.) I need to include it in within the values.yaml file?

Based on the information provided in the extracts, it's not explicitly mentioned which specific pods require the Cloud SQL proxy as a sidecar for connecting to an external Cloud SQL database. However, from the discussions, it's clear that the database connection is crucial for the functioning of Airbyte, so it's reasonable to assume that the

db

pod might require it. In one of the [Slack conversations](https://airbytehq.slack.com/archives/C021JANJ6TY/p1666160810163809), a user mentioned that the

webapp

pod does not need to access the database directly as it makes requests to the server, and then the server accesses the database. So, it might not require the Cloud SQL proxy sidecar. For a definitive answer, I would recommend checking the official Airbyte documentation or reaching out to the Airbyte community on their [Slack channel](https://airbytehq.slack.com/archives/C021JANJ6TY/p1666160810163809) or [discussion forum](https://discuss.airbyte.io/). They should be able to provide more specific guidance based on your exact setup and requirements.

@kapa.ai do you have any examples of the Cloud SQL proxy being defined as a sidecar within the values.yaml file?

I don’t have full knowledge on which pods all make direct db calls to but I would start with server and db, as I think for sure those do. I think it should be fairly obvious after that if something fails at least

I was able to resolve the connection errors I was encountering with the bootloader by changing the

jdbcUrl

attribute in the

externalDatabase

section from

"jdbc:<postgresql://cloudsqlproxy:5432/airbyte-db?ssl=true&sslmode=require>"

to:

"jdbc:<postgresql://cloudsqlproxy:5432/airbyte-db>"

i.e. removing the parameters. Once I removed these, I no longer saw any database connection errors during the bootloader process. Note I also needed to deploy the Cloud SQL proxy as a separate pod within the cluster, rather than as a sidecar within the bootloader pod. I also configured a K8s service (named

cloudsqlproxy

, hence the host in the argument value above). Unfortunately although deploying the Cloud SQL proxy as a sidecar within the bootloader worked in terms of allowing the bootloader container to complete, the fact the proxy pod continued running meant that the pod wouldn't complete, and thus the remainder of the installation could not proceed.

makes sense. The bootloader itself would ideally be some kubernetes job pod and would terminate alongside the sidecar, but we purposefully chose not to make it a job pod because there were no guarantees at the time that duplicates would not be created. I think setting it up as a long lived container makes sense and is fine

Thanks @Conor Barber (Airbyte), appreciate the input!

I had the same issue with bootloader using cloud-sql-proxy as a sidecar. I am currently working around it by leaving bootloader disabled (after the initial run \ migration). That said, it would be nice if we could configure the bootloader to have an additional argument to run a specific command when its complete. For example, cloud-sql-proxy can have an api endpoint running that will terminate if its called (/quitquitquit) Or if shareProcessNamespace is true, then you can use pkill. Another simpler but hackier option would be to change the bootloader job manifest to include

shareProcessNamespace: true

. Then we could just deploy another sidecar that, for example, sleeps for 60s and then runs

pkill -SIGTERM cloud-sql-proxy

.