My feedback is this: Our Airbyte instance stopped working unexpectedly yesterday. Today is the second day that we are unable to deploy a fresh Airbyte instance on an EC2 machine using an external database (AWS RDS). We are consistently encountering the error: “Airbyte Bootloader failed to start.” This situation is highly disruptive, and it seems that every time we need to make changes or updates in Airbyte, it results in 1–2 days of downtime. At this point, it raises serious concerns about the reliability of the tool for production use.

What version of Airbyte are you running? And helm v1 or v2? Also, is there anything in the bootloader logs? You said "stopped working" . . . was there any action that was correlated to this? (e.g. installed an update, database maintenence, new connection added, etc.)

@Justin Beasley We've been using an early version (1.0.0 or 1.0.1, I’m not sure which). The existing EC2 instance hasn’t changed. We're now trying to deploy a new Airbyte instance (1.8.5 and 2.0.0) on another EC2 machine with an external RDS, but it’s not working yet.

We're on 1.7.7 on a GKE Autopilot cluster on GCP (using full Google, Cloud SQL/Cloud Storage/Cloud LB/Secret Manager/IAP for Google Account login/etc.). We're almost entirely BigQuery destinations. I haven't had time to work through the changes in 1.8/2.0 since I'm wanting to update to helm v2 and need to test through that more. But for new installs this shouldn't be too painful. Often people having issues on RDS specifically are related to issues with the connections to RDS not being allowed, or requiring a certificate that isn't available in the workload. Might be worth checking logs for things related to that. The bootloader logs are very likely to have errors related to DB connectivity in that case.

Thanks for that information 🙂. Just out of curiosity is there a documentation on how to set up Airbyte in GKE or EKS?

The pieces are largely here: https://docs.airbyte.com/platform/deploying-airbyte . . . but some assembly required since there are a lot of different ways you might want to set things up

@Justin Beasley Thanks. Most probably we will switch from EC2 deployment to EKS, I think it will be more reliable to use managed Kubernetes cluster instead of Kubernetes in Docker.

It's interesting that the boot loader is failing on a fresh install. We've probably tested a fresh install dozens of times at this point on all kinds of machines. My guess is the same as Justin's - a connectivity issue to the DB or perhaps some other external service. Getting those logs from the pod is going to be key. You can run them through the #C01AHCD885S channel too if you want. If kapa's seen the problem before it might see the issue right away. You also mentioned you're upgrading about 10 versions at once. I'm guessing you're using abctl? It might be worth checking that you're running the latest version of that tool also. From time to time a new Airbyte version requires a certain minimum abctl version. https://docs.airbyte.com/release_notes/v-2.0#abctl-0302-required

I'll only echo the difficulties in working with the new charts vs the old docker compose process 🙂

Hey @Justin Beasley have you upgraded to 2.0? Just wanna point out (since your deployment is very similar to mine) some things that broke when I upgraded (1.7.2 -> 2.0). Also using the v2 chart (2.0.19) • undocumented prop

global.storage.gcs.credentialsJsonSecretKey

when translating v1 logging storage config. Docs only show(ed when I upgraded) a way to configure the actual secret on values.yaml • had problems with database

global.database

fields, where I think temporal was trying to deploy to the airbyte database instead of its own. Had to add temporal environment variables so that it would point to the correct databases (

temporal

and

temporal_visibility

) • (still not solved) after the upgrade, I'm seeing a lot of temporal logs

io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: namespace rate limit exceeded... leads me to think I'll have to tinker with temporal variables

Also, after I got everything working, some of my (many, lol) builders broke, specifically on the pagination section Issue was: • suddenly, the "inject into" body or header all vanished • after I re-added them, the tests failed with a gigantic stacktrace • found out that after re-adding the fields, the builder didn't "clean" the old

paginator.page_size_option.field_name

field and added the new

paginator.page_size_option.field_path[*]

field. This is against the spec, since only one or the other should be specified • had to manually edit the yaml removing the old field can dm the final

values.yaml

if you want

@Lucas Segers I haven't made the jump yet due to some other deadlines I've been working on and needing to test this more with the new charts—so this is super helpful. I'd love if you can pass along your final values.yaml! I'll try to do some experimentation in a test clone of my cluster and see if I can find any workarounds when I do (probably still a couple weeks out for me though)

my values.yaml went something like this, nothing too fancy

postgresql:
  enabled: false
minio:
  enabled: false
serviceAccount:
  create: true
  annotations:
    <http://iam.gke.io/gcp-service-account|iam.gke.io/gcp-service-account>: integration@MY_PROJECT.iam.gserviceaccount.com
  name: airbyte-admin

server:
  replicaCount: 2
  resources:
    limits:
      cpu: 1000m
      memory: 2Gi
    requests:
      cpu: 150m
      memory: 1Gi

worker:
  replicaCount: 2
  extraEnv:
    - name: DISCOVER_REFRESH_WINDOW_MINUTES
      value: "0"

temporal:
  replicaCount: 1
  extraEnv:
    - name: DBNAME
      value: "temporal"
    - name: VISIBILITY_DBNAME
      value: "temporal_visibility"
    - name: SKIP_DB_CREATE
      value: "true"
    - name: POSTGRES_TLS_ENABLED
      value: "false"
    - name: SQL_TLS_ENABLED
      value: "false"

global:
  serviceAccountName: airbyte-admin
  edition: community
  local: false
  airbyteUrl: "<https://MY_INGRESS_URL.mydomain.com>"

  auth:
    enabled: false
  ingress:
    enabled: false
  metrics:
    enabled: false

  database:
    type: external
    name: airbyte
    host: sql-proxy-service
    port: 3000
    password: "a" # fake, we use cloud-sql-proxy
    user: integration@MY_PROJECT.iam

  storage:
    type: gcs
    secretName: ab-storage-v3
    bucket:
      activityPayload: MY_PROJECT-airbyte
      log: MY_PROJECT-airbyte
      state: MY_PROJECT-airbyte
      workloadOutput: MY_PROJECT-airbyte
    gcs:
      projectId: MY_PROJECT
      credentialsJsonSecretKey: gcp.json

  workloads:
    resources:
      useConnectorResourceDefaults: true
      mainContainer:
        cpu:
          request: 100m
          limit: 1000m
        memory:
          request: 250Mi
          limit: 1Gi
      check:
        cpu:
          request: 100m
          limit: 1000m
        memory:
          request: 250Mi
          limit: 1Gi
      discover:
        cpu:
          request: 100m
          limit: 1000m
        memory:
          request: 250Mi
          limit: 1Gi
      replication:
        cpu:
          request: 100m
          limit: 1000m
        memory:
          request: 250Mi
          limit: 1Gi
      sidecar:
        cpu:
          request: 100m
          limit: 1000m
        memory:
          request: 250Mi
          limit: 1Gi

@Lucas Segers why did you set this as true:

useConnectorResourceDefaults: true

but also provided custom resources and limits?

hiii @Yuvaraj Prem Kumar I have some connectors that I set the resources directly on them so the idea (from what I understood) is: • values.yaml as default • connector resource defaults (the ones we set on the connector table) override this

Thanks @Lucas Segers. I am trying to follow your values.yaml, did you get this error? I am using the airbyte v2 helm chart

Deployment.apps "airbyte-worker" is invalid: spec.template.spec.containers[0].env[66].valueFrom: Invalid value: "": may not be specified when `value` is not empty
helm.go:92: 2025-11-20 08:57:38.760297262 +0000 UTC m=+89.822591872 [debug] cannot patch "airbyte-worker" with kind Deployment: Deployment.apps "airbyte-worker" is invalid: spec.template.spec.containers[0].env[66].valueFrom: Invalid value: "": may not be specified when `value` is not empty

The error seems to come from

worker:
  extraEnv:
    - name: DISCOVER_REFRESH_WINDOW_MINUTES
      value: "0"

hey I didn't get this error when applying the chart this env is for disabling the automatic schema discovery

hi, yes I know what the setting is used for. I also wanted to disable it , in order to save some execution time. But the latest v2 helm chart version does not allow for this extraEnv it seems...

I went with 2.0.19

Ok strange, mine is the same

its this one, I just replaced my project info

Thanks. This is mine but it just does not work

worker:
  enabled: true
  replicaCount: 3
  serviceAccountName: "airbyte"
  resources:
    requests:
      cpu: 1000m
      memory: 2Gi
    limits:
      cpu: 2000m
      memory: 4Gi
  maxCheckWorkers: 10
  maxSyncWorkers: 10
  hpa:
    enabled: false
  log:
    level: "INFO"
  extraEnv:
    - name: DISCOVER_REFRESH_WINDOW_MINUTES
      value: "0"