Is there any documentation on how to implement Octavia for k8s deployments of Airbyte?

This is a good question that I’ve asked for awhile, but there doesn’t seem to be a straightforward way. I’ve done it. It isn’t pretty but it works. My infra doesn’t allow local connection to my dev/prod database, so I’m using alot of kubectl commands to pull/push files to the octavia container. An overview: 1. Alongside your Airbyte deployment, deploy a K8s Job that uses an octavia-cli docker image. The job itself is always running using the command

while true; do sleep 30; done;

2. Set an environment variable for your

AIRBYTE_URL

to be pointed at your web svc. In my case, it’s http://airbyte-airbyte-webapp-svc:80. 3. Exec into the container to run your octavia cli commands against your deployed Airbyte service. 4. Because of my limitations, all files that I either apply, or retrieve, have to be done using the container as a conduit, so I have kubectl commands to copy files from local to container and vice versa. Hopefully that helps.

You can use a kubernetes job based on octavia docker image to deploy your airbyte conf

@Dusty Shapiro Thanks so much for the detailed answer! This isn’t a super high priority for us atm, but I plan on coming back to this comment to implement!

It isn’t a huge feature we rely on - but I wanted the ability to check configurations into version control just incase our app DB blows up.

Precisely, plus I’m finding that I’ve had to replicate connections/create similar connections/etc. It seems at the present moment, it’s best to create many connections with a few streams rather than one stream with many connections, which can be time consuming via the GUI.

I think the official helm chart needs to be shipped with the octavia-cli

@Vincent Koc could you elaborate a bit? I’m unfamiliar with managing helm

Helm is used to make kubernetes configurable. We need to ship octaviacli as a pod and have it all setup so it can interact with Airbyte main worker/web app. This will need some work to get the helm chart stable so its easy to install and use.