https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> we are migrating from 1.5.0 to 2.0....
# replication-ask-ai
g
@kapa.ai we are migrating from 1.5.0 to 2.0.0, I have an error Encountered an issue deploying Airbyte: Pod: airbyte-abctl-bootloader.1899a2d7da2bed57 Reason: Failed Message: Error: couldn't find key DATABASE_USER in Secret airbyte-abctl/airbyte-config-secrets Count: 6 any idea
k
This error happens because in 2.0.0 the bootloader (and other pods) expect certain keys to exist in the 
airbyte-config-secrets
secret, and they’re missing in your cluster. From the error: > couldn't find key 
DATABASE_USER
in Secret 
airbyte-abctl/airbyte-config-secrets
you need to ensure that the secret in the 
airbyte-abctl
namespace contains the expected database keys. For 2.x, the recommended pattern is: 1. Create/update the 
airbyte-config-secrets
secret in the same namespace where you’re deploying (here: 
airbyte-abctl
):
Copy code
apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
  namespace: airbyte-abctl
type: Opaque
stringData:
  # Database Secrets
  database-host: <your-db-host>
  database-port: "5432"
  database-name: <your-db-name>
  database-user: <your-db-user>
  database-password: <your-db-password>
You can also create it via 
kubectl create secret generic airbyte-config-secrets ... --namespace airbyte-abctl
. [K8s secret] 2. In your 
values.yaml
, configure the external database to read from that secret (Helm v2 style):
Copy code
postgresql:
  enabled: false

global:
  database:
    type: external
    secretName: airbyte-config-secrets
    host: "<your-db-host>"
    port: "5432"
    name: "<your-db-name>"
    userSecretKey: "database-user"
    passwordSecretKey: "database-password"
[External DB 2.0] 3. Re‑apply the secret (if changed) and then re‑run 
abctl
/ Helm so the bootloader pod is recreated. If after this you still see 
DATABASE_USER
missing, double‑check: • The namespace of the secret matches the namespace in the error (
airbyte-abctl
). • The secret name is exactly 
airbyte-config-secrets
(or, if you changed it, that 
global.database.secretName
matches). • The keys are spelled exactly as expected (
database-user
, 
database-password
, etc.). [External DB] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai i have another one now Failed to install airbyte/airbyte Helm Chart ERROR Unable to install Airbyte locally ERROR unable to install airbyte chart: unable to install helm: pre-upgrade hooks failed: 1 error occurred: * pod airbyte-abctl-bootloader failed
k
This specific error (
pre-upgrade hooks failed: * pod airbyte-abctl-bootloader failed
) is a generic symptom: it only tells us the bootloader pod failed, not why. The Airbyte docs explicitly say that the root cause is usually that the bootloader cannot connect to 
airbyte-db
or is missing required config, and that you must inspect the bootloader pod logs to see the real error. [Troubleshoot abctl] From the docs: > The 
airbyte-bootloader
is the first to start during the installation. Version 
>0.15.0
of 
abctl
prints the service logs to make simpler to understand what is causing the issue. Failures reasons are generally related to problem of 
airbyte-bootloader
not able to connect to the 
airbyte-db
service. [Troubleshoot abctl] Given you just fixed a missing secret key (
DATABASE_USER
), it’s very likely the bootloader is now failing on a different missing key or DB connectivity. Please do this and paste the result: 1. Get the bootloader pod name (if you don’t already have it):
Copy code
kubectl get pods -n airbyte-abctl | grep bootloader
2. Show its logs:
Copy code
kubectl logs -n airbyte-abctl <bootloader-pod-name>
abctl
v0.15.0+ should also print these logs automatically during install; look for lines like:
Copy code
DEBUG   looking at airbyte-abctl-airbyte-bootloader
        Failed()
DEBUG   found logs: ...
Those log lines (the text after 
found logs:
) will contain the real error (for example, another missing secret key like 
DATABASE_PASSWORD
, or a DB connection error), similar to what’s shown in several GitHub issues where the same 
pre-upgrade hooks failed
message appears. [Issue 54199; Troubleshoot abctl] Once you share that bootloader log snippet, we can pinpoint the exact next fix. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
ok @kapa.ai but i use external db, here is my values.yaml global: # logging: # level: INFO auth: cookieSecureSetting: "false" database: type: external host: "{{ airbyte_database_host|lower }}" port: "{{ airbyte_database_port }}" database: "{{ airbyte_database }}" user: "{{ airbyte_database_user }}" secretName: "airbyte-config-secrets" passwordSecretKey: "database-password" worker: maxNotifyWorkers: 1 postgresql: enabled: false externalDatabase: host: "{{ airbyte_database_host|lower }}" port: "{{ airbyte_database_port }}" database: "{{ airbyte_database }}" user: "{{ airbyte_database_user }}" existingSecret: "airbyte-config-secrets" existingSecretPasswordKey: "database-password" jdbcUrl: "jdbcpostgresql//{{ airbyte_database_host|lower }}:{{ airbyte_database_port }}/{{ airbyte_database }}" temporal: externalSSL: false extraEnv: - name: SKIP_DB_CREATE value: "true" - name: POSTGRES_SEEDS value: "{{ airbyte_database_host|lower }}" - name: DB_PORT value: "{{ airbyte_database_port }}" - name: POSTGRES_USER value: "userairbyte" - name: POSTGRES_PWD value: "userairbyte" - name: POSTGRES_TLS_ENABLED value: "false" - name: POSTGRES_TLS_DISABLE_HOST_VERIFICATION value: "true" - name: SQL_TLS_ENABLED value: "false" - name: SQL_TLS_DISABLE_HOST_VERIFICATION value: "true"
k
You’re very close; the issue is that your 
values.yaml
is still using Helm chart V1 keys for the external DB, but you’re deploying with chart V2 / 2.0.0. In V2: 1. The DB name key changed from 
database
→ 
name
. 2. The recommended pattern is to use either `user`/`password` or `userSecretKey`/`passwordSecretKey`, not a mix. 3. The old 
externalDatabase:
block is no longer used; configuration is done via 
global.database
only. [External DB 2.0; Upgrade values.yaml] For Helm chart V2 / 2.0.0, your 
values.yaml
should look like this (for external Postgres):
Copy code
postgresql:
  enabled: false

global:
  database:
    type: external

    # Secret with DB credentials
    secretName: "airbyte-config-secrets"

    # External Postgres connection
    host: "{{ airbyte_database_host|lower }}"
    port: "{{ airbyte_database_port }}"
    name: "{{ airbyte_database }}"          # <-- was `database` in V1

    # Use EITHER user or userSecretKey, not both
    user: "{{ airbyte_database_user }}"
    # userSecretKey: "database-user"

    # Use EITHER password or passwordSecretKey, not both
    # password: "<plain-password-if-you-want-it-here>"
    passwordSecretKey: "database-password"
And you can remove the whole 
externalDatabase:
section; it’s a V1 construct and not needed in V2. [External DB 2.0; Upgrade values.yaml] Also ensure your secret matches this:
Copy code
apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
  namespace: airbyte-abctl
type: Opaque
stringData:
  database-password: "<your-db-password>"
  # (optional if you use userSecretKey)
  database-user: "<your-db-user>"
After updating: 1. Apply/update the secret. 2. Re-run 
abctl local install --values values.yaml
(or your Helm install). 3. If it still fails, grab the bootloader pod logs again; but with the corrected 
global.database
block and no 
externalDatabase:
section, it should pass the pre-upgrade hook. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai how to know if i am using v2 or v1
k
You can tell from how you install/upgrade and which repo/chart name you use: 1. Check the Helm repo & chart name ◦ V1 uses the old repo path and chart:
Copy code
helm repo add airbyte https://airbytehq.github.io/helm-charts
       helm install airbyte airbyte/airbyte ...
◦ V2 uses the new 
/charts
repo and 
airbyte-v2/airbyte
chart:
Copy code
helm repo add airbyte-v2 https://airbytehq.github.io/charts
       helm search repo airbyte-v2 --versions
       helm install airbyte airbyte-v2/airbyte ...
[Chart v2 community; Install 2.0] If your 
helm install/upgrade
command uses 
airbyte-v2/airbyte
, you’re on *Helm chart V2*; if it uses 
airbyte/airbyte
from the old repo, you’re on V1. 2. Check existing release Run:
Copy code
helm list -n <NAMESPACE>
Then inspect the chart for that release:
Copy code
helm get all airbyte -n <NAMESPACE> | grep 'chart='
◦ If it shows 
airbyte-v2/airbyte-2.x.x
, that’s V2. ◦ If it shows `airbyte-`version from repo 
airbyte
, that’s V1. [Upgrading Airbyte] 3. abctl hint 
abctl
0.30.0+ uses Helm chart V2 by default for Core. If you installed with a recent 
abctl
and didn’t override the chart, you’re likely on V2. [Chart v2 community] If you paste your exact 
helm install
/ 
abctl
command or 
helm list
output, I can tell you definitively which one you’re using. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
ok @kapa.ai now there is an issue with kind I think ERROR Failed to install airbyte/airbyte Helm Chart ERROR Unable to install Airbyte locally ERROR unable to install airbyte chart: unable to install helm: cannot patch "airbyte-abctl-connector-builder-server" with kind Deployment: Deployment.apps "airbyte-abctl-connector-builder-server" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"airbyte":"connector-builder-server", "app.kubernetes.io/instance":"airbyte-abctl", "app.kubernetes.io/name":"connector-builder-server"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "airbyte-abctl-cron" with kind Deployment: Deployment.apps "airbyte-abctl-cron" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"airbyte":"cron", "app.kubernetes.io/instance":"airbyte-abctl", "app.kubernetes.io/name":"cron"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && failed to create patch: The order in patch list:
k
This error is not caused by 
kind
itself; it’s a known issue when upgrading an existing `abctl`/Helm deployment where the Deployment label selectors changed, and Kubernetes does not allow changing 
spec.selector
on an existing Deployment (it’s immutable). The exact error you see matches an open GitHub issue. [Helm abctl issue] Because the chart is trying to patch existing Deployments (
airbyte-abctl-connector-builder-server
, 
airbyte-abctl-cron
, etc.), the upgrade fails. The knowledge sources do not provide an official step‑by‑step fix for this specific selector‑immutability problem. They only state: • 
abctl
0.30.0+ uses Helm chart V2 by default. [Chart v2 upgrade] • For Helm chart V2, the recommended path is a fresh install in a new namespace when possible, to avoid exactly these kinds of upgrade edge cases. [How to upgrade] Given that, the documented and supported workaround is: 1. Prefer a fresh install in a new namespace (recommended in the docs for V2): ◦ Create a new namespace, e.g.:
Copy code
kubectl create namespace airbyte-v2
◦ Configure your external DB and secrets as you already did. ◦ Install Airbyte into this new namespace using the V2 chart / 
abctl
. [How to upgrade] 2. If you must stay in the same namespace, the docs do not describe a supported way to mutate those existing Deployments’ selectors in place. Because 
spec.selector
is immutable, Helm cannot patch them, which is exactly what your error shows. [Helm abctl issue] So, based on the available information: • This is a known upgrade problem with 
abctl
+ Helm chart changes. • The documented, supported path is to: ◦ Use an external DB (which you already do), ◦ Then perform a fresh Helm V2 / abctl install in a new namespace, and point it at the same external DB and storage. [Chart v2 upgrade] The knowledge sources do not contain more granular remediation steps (like specific 
kubectl
commands to delete or recreate those Deployments), so I can’t provide an official in‑place fix beyond recommending the fresh‑namespace approach. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai whats the newest version that is compatible with V1
k
The knowledge sources state that Airbyte 2.0 is the final version that supports Helm chart V1. [Airbyte 2.0 notes] So, the newest Airbyte version compatible with Helm chart V1 is 2.0.x (specifically mentioned as 2.0 in the docs). Versions 2.1 and later only support Helm chart V2. [Airbyte 2.0 notes] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai how to force the use of helm chart v1
k
To force the use of Helm chart V1, you need to: 1. Use the V1 Helm repo and chart name Add the V1 repo and install/upgrade from it:
Copy code
helm repo add airbyte https://airbytehq.github.io/helm-charts
   helm repo update
Then install/upgrade using the V1 chart:
Copy code
helm install airbyte airbyte/airbyte \
     --namespace airbyte \
     --values ./values.yaml \
     --version <V1_HELM_CHART_VERSION>
or, for upgrade:
Copy code
helm upgrade --install airbyte airbyte/airbyte \
     --namespace airbyte \
     --values ./values.yaml \
     --version <V1_HELM_CHART_VERSION>
[Add V1 repo; Upgrade with V1] 2. Avoid the V2 repo/chart Do not use:
Copy code
helm repo add airbyte-v2 https://airbytehq.github.io/charts
   helm install airbyte airbyte-v2/airbyte ...
Those commands explicitly select Helm chart V2. [Add V2 repo] 3. Stay on an Airbyte version that still supports V1 Airbyte *2.0 is the final version that supports Helm chart V1*; 2.1+ require V2. So if you want to keep using V1, you must stay on ≤ 2.0 and pick a matching V1 chart version from the 
airbyte/airbyte
repo. [Final V1 support] If you share your current 
helm list -n
namespace output, I can suggest an exact 
--version
value for the V1 chart. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai I use abctl, helm is not installed
k
With 
abctl
you can force Helm chart V1 by pinning the chart version via 
--chart-version
. From the maintainer’s comment: > abctl versions 0.30.0 and later support deploying Airbyte version 1.8+ using helm chart v2. However, you should still be fine to deploy with chart V1. Use the 
--chart-version
flag to specify the chart you want to use. > For example: > 
abctl local install --chart-version 1.9.1
for Airbyte 2.0. [abctl chart flag] So, to force V1 with `abctl`:
Copy code
abctl local uninstall          # optional, if you want a clean reinstall
abctl local install --chart-version 1.9.1 --values values.yaml
(or another specific V1 chart version you want to stay on). Key points: • 
abctl
is just wrapping Helm; 
--chart-version
tells it exactly which Helm chart (V1) to use. [abctl reference] • Airbyte 2.0 is the last Airbyte version that supports Helm chart V1. [Final V1 support] If you share the Airbyte version you’re targeting, you can pick a matching V1 chart version and pass it via 
--chart-version
. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai i do this 
abctl local install --chart-version 2.0.0
and helm v2 is used, how to force v1
k
--chart-version 2.0.0
is a V2 Helm chart version, so 
abctl
is correctly using Helm chart V2 in that case. To force Helm chart V1 with 
abctl
, you must pass a V1 chart version to 
--chart-version
. From the maintainer’s guidance: > abctl versions 0.30.0 and later support deploying Airbyte version 1.8+ using helm chart v2. However, you should still be fine to deploy with chart V1. Use the 
--chart-version
flag to specify the chart you want to use. > For example, 
abctl local install --chart-version 1.9.1
for Airbyte 2.0. [Issue 68136] So, to force V1, use a 1.x.y chart version, e.g.:
Copy code
# Example: force Helm chart V1 for Airbyte 2.0
abctl local install --chart-version 1.9.1
Any 
1.*
chart version is from the V1 repo; 
2.*
chart versions are V2. The knowledge sources don’t list all valid V1 versions, but they explicitly show 
1.9.1
as a V1 chart version usable for Airbyte 2.0. [Issue 68136] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai i switched to 1.9.1 and now I have ERROR Failed to install airbyte/airbyte Helm Chart ERROR Unable to install Airbyte locally ERROR unable to install airbyte chart: unable to install helm: cannot patch "airbyte-abctl-workload-api-server-svc" with kind Service: Service "airbyte-abctl-workload-api-server-svc" is invalid: spec.ports[1].name: Duplicate value: "http" && failed to create patch: The order in patch list: [map[name:BIND_ON_IP value:0.0.0.0] map[name:TEMPORAL_BROADCAST_ADDRESS value:0.0.0.0] map[name:POSTGRES_SEEDS value:stel-rec-datdb.texa.frsi] map[name:POSTGRES_SEEDS valueFrommap[configMapKeyRefmap[key:DATABASE_HOST name:airbyte-abctl-airbyte-env]]] map[name:DB_PORT value:5432] map[name:DB_PORT valueFrommap[configMapKeyRefmap[key:DATABASE_PORT name:airbyte-abctl-airbyte-env]]] map[name:POSTGRES_USER valueFrommap[secretKeyRefmap[key:database-user name:airbyte-config-secrets]]] map[name:POSTGRES_USER value:userairbyte] map[name:POSTGRES_PWD valueFrommap[secretKeyRefmap[key:database-password name:airbyte-config-secrets]]] map[name:POSTGRES_PWD value:userairbyte] map[name:SQL_TLS_ENABLED value:false] map[name:SQL_TLS_ENABLED value:true]] doesn't match $setElementOrder list: [map[name:AUTO_SETUP] map[name:DB] map[name:BIND_ON_IP] map[name:TEMPORAL_BROADCAST_ADDRESS] map[name:DYNAMIC_CONFIG_FILE_PATH] map[name:POSTGRES_SEEDS] map[name:DB_PORT] map[name:POSTGRES_USER] map[name:POSTGRES_PWD] map[name:POSTGRES_TLS_ENABLED] map[name:POSTGRES_TLS_DISABLE_HOST_VERIFICATION] map[name:SQL_TLS_ENABLED] map[name:SQL_TLS_DISABLE_HOST_VERIFICATION] map[name:SKIP_DB_CREATE] map[name:POSTGRES_SEEDS] map[name:DB_PORT] map[name:POSTGRES_USER] map[name:POSTGRES_PWD] map[name:POSTGRES_TLS_ENABLED] map[name:POSTGRES_TLS_DISABLE_HOST_VERIFICATION] map[name:SQL_TLS_ENABLED] map[name:SQL_TLS_DISABLE_HOST_VERIFICATION] map[name:AIRBYTE_INSTALLATION_ID]]
k
The knowledge sources describe this exact class of error as a known, still‑being‑fixed problem in `abctl`/Helm when patching resources: the patch contains duplicated/contradictory env vars and/or fields, so Helm can’t apply it and you see messages like: • 
Service ... is invalid: spec.ports[1].name: Duplicate value: "http"
• 
failed to create patch: The order in patch list ... doesn't match $setElementOrder list
[Patch order error; Issue 60918] In your log, you can see the duplicates clearly: • 
POSTGRES_SEEDS
appears twice (literal value + 
valueFrom
). • 
DB_PORT
appears twice. • 
POSTGRES_USER
appears twice. • 
POSTGRES_PWD
appears twice. • 
SQL_TLS_ENABLED
appears twice with conflicting values (
false
and 
true
). This is the same pattern as the open GitHub issue where 
abctl local install
fails because the generated patch’s env var list doesn’t match the expected 
$setElementOrder
list. [Issue 60918] Unfortunately, the provided docs and issues: • Acknowledge the bug (
failed to create patch: The order in patch list
) and mark it as “Status: Fixing”. [Patch order error] • Do not document a reliable workaround for when you hit it during `abctl local install`/upgrade, beyond indicating it happens when re‑running installs with different modes or configurations (e.g. low‑resource mode, env changes). [Patch order error; Issue 60918] Given that limitation, the only actions that are clearly supported in the knowledge sources are: 1. Avoid repeated, conflicting installs in the same local cluster/namespace. The docs note this error can occur when you install with one set of options and then re‑run 
abctl local install
with different options (e.g. 
--low-resource-mode
), which leads to incompatible patches. [Patch order error] 2. Start from a clean local environment (new kind cluster / namespace) and reinstall. While not spelled out step‑by‑step for this exact error, the troubleshooting guidance for similar Helm/patch issues with 
abctl
implies that a fresh install (no existing Airbyte resources to patch) avoids the patch‑order and immutability problems. [Patch order error; Issue 60918] Because the sources do not provide a documented way to “fix” the duplicate env vars / port names inside the generated patch or to safely mutate those existing resources in place, I can’t give you an official in‑place remediation. Based on the docs and issues we have: • This is a known 
abctl
+ Helm patch bug. • The supported path to get past it is to: ◦ Tear down the existing local deployment (or create a new kind cluster / namespace), and ◦ Re‑run 
abctl local install
with your desired chart version and values, so Helm does a clean install instead of trying to patch existing Services/Deployments. [Patch order error; Issue 60918] If you need to keep the current cluster, the knowledge sources don’t describe a safe, supported sequence of 
kubectl
operations to resolve this, so I can’t recommend one based on the available documentation. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai what do you think, should we go straight to 2.0.x or 1.9.x
k
Based on the docs and release notes, the main decision point is Helm chart V1 vs V2: • Airbyte 2.0.x is the final version that supports Helm chart V1. 2.1+ require Helm chart V2 only. [Airbyte 2.0 notes] • New deployments are recommended to use Helm chart V2 from the start to avoid another migration later. [Airbyte 2.0 notes; Chart v2 core] • Existing deployments are strongly encouraged to migrate to chart V2 “as soon as practical” so you’re not blocked from future upgrades. [Airbyte 2.0 notes] Given you’re already doing a significant jump (1.5.0 → 2.0.0) and have hit several chart/abctl issues, the docs implicitly favor: • Targeting 2.0.x with Helm chart V2, and • Doing the recommended V2 migration pattern: external DB + fresh install in a new namespace, then cut over. [Chart v2 core; Chart v2 enterprise] Staying on 1.9.x (Airbyte 1.9 with chart V1) would only make sense if you want a short‑term, more conservative step and are okay with doing another upgrade/migration soon after, since you’ll still need to move to chart V2 before going beyond 2.0. [Chart v2 core; Airbyte 2.0 notes] So, from the documentation’s perspective: if you can tolerate doing the migration work now, go straight to 2.0.x with Helm chart V2 rather than stopping at 1.9.x. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai switching to 2.0.19, I cleaned everything and I have this _ _ _ _ / | (_)____/ /_ __ __/ /____ / /| | / / ___/ __ \/ / / / __/ _ \ / _ |/ / / / /_/ / /_/ / /_/ __/ /_/ |_/_/_/ /_.___/\__, /\__/\___/ /____/ : airbyte-workload-launcher : 2026-03-04 151322,990 [main] INFO i.m.c.e.DefaultEnvironment(<init>):170 - Established active environments: [k8s, control-plane, edition-community] 2026-03-04 151323,729 [main] INFO c.z.h.HikariDataSource(<init>):79 - config-pool - Starting... 2026-03-04 151323,775 [main] INFO c.z.h.HikariDataSource(<init>):81 - config-pool - Start completed. 2026-03-04 151323,964 [main] INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.netty' 2026-03-04 151323,965 [main] INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'ERROR' for logger: 'com.zaxxer.hikari' 2026-03-04 151323,965 [main] INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.grpc' 2026-03-04 151323,965 [main] INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'ERROR' for logger: 'com.zaxxer.hikari.pool' 2026-03-04 151323,965 [main] INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'INFO' for logger: 'io.fabric8.kubernetes.client' 2026-03-04 151324,861 [scheduled-executor-thread-2] INFO i.a.w.l.PodSweeper(sweepPods):56 - Starting pod sweeper cycle in namespace [airbyte-abctl]... 2026-03-04 151324,870 [scheduled-executor-thread-2] INFO i.a.w.l.PodSweeper(sweepPods):88 - Will sweep Succeeded pods older than 2026-03-04T150324.862Z (UTC). 2026-03-04 151324,871 [scheduled-executor-thread-2] INFO i.a.w.l.PodSweeper(sweepPods):91 - Will sweep unsuccessful pods older than 2026-03-04T131324.862Z (UTC). 2026-03-04 151325,935 [scheduled-executor-thread-1] ERROR i.a.a.c.a.AccessTokenInterceptor(intercept):120 - Failed to obtain or add access token
k
The log shows 
workload-launcher
is up but failing on: > 
ERROR ... AccessTokenInterceptor(intercept):120 - Failed to obtain or add access token
This is a known class of issues in 2.x where components cannot get a token for the workload API / data plane, often due to missing or misconfigured auth secrets or env vars. [Dataplane token 500; Workload auth 401; Queued/workload never claimed] However, the knowledge sources do not provide a complete, 2.0.19‑specific, abctl‑based fix for this exact 
AccessTokenInterceptor
error. They only show related patterns: • For older 1.x “Failed to obtain or add dataplane access token”, some users deleted 
dataplane_*
DB rows or removed 
airbyte-auth-secrets
, but that’s not documented as a 2.0 solution. [Dataplane token 500] • For workload API auth, users had to ensure: ◦ 
API_AUTHORIZATION_ENABLED=true
in the 
airbyte-airbyte-env
ConfigMap. ◦ Matching 
WORKLOAD_API_AUTH_HEADER
and 
WORKLOAD_API_BEARER_TOKEN
on the workload API server and worker. [Workload auth 401; HTTP 502 connectors; Local abctl bearer token; HTTP 500 workaround note] Given that, the only concrete, source‑backed steps I can suggest are: 1. Check if this is workload API auth related Look at the 
workload-launcher
logs a bit further down for an HTTP status (401/500) or messages about “workload API” or “data-plane”. The issues above show that 
AccessTokenInterceptor
failures often wrap a 401/500 from the internal API. [Queued/workload never claimed] 2. Verify workload API auth env vars (if applicable) For Kubernetes/abctl deployments, users who hit 401/502 errors on connector setup had to align these env vars across components: ◦ 
WORKLOAD_API_AUTH_HEADER
◦ 
WORKLOAD_API_BEARER_TOKEN
For example, in abctl local installs, they manually set the same token on worker and workload API server and restarted them. [HTTP 502 connectors; Local abctl bearer token] You can inspect what 
workload-launcher
has:
Copy code
kubectl -n airbyte-abctl get deployment airbyte-workload-launcher -o json \
     | jq -r '.spec.template.spec.containers[0].env[] | select(.name | contains("WORKLOAD"))'
If 
WORKLOAD_API_BEARER_TOKEN
is missing or doesn’t match the workload API server, that could explain token failures, but the sources don’t show a 2.0.19‑specific fix path for 
workload-launcher
itself. 3. Acknowledge limitation The knowledge sources do not explicitly document how 
workload-launcher
in 2.0.19 is supposed to obtain its access token (e.g., which secret or endpoint), nor a step‑by‑step remediation for this exact 
AccessTokenInterceptor
error in 2.0.19. Because of that, I can’t confidently give you a precise fix beyond checking workload API auth configuration as above. If you can share the full stack trace after 
Failed to obtain or add access token
(especially any 
Caused by: ... HTTP 401/500
lines), I can try to match it more closely to one of the documented issues, but right now the sources don’t contain enough detail to give a definitive resolution. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai
Copy code
abctl local install --host HOST \
    --insecure-cookies --no-browser \
    --chart-version 2.0.19 \
    --secret /secret.yaml \
    --values /values.yaml \
    --docker-username USERNAME --docker-password PASSWORD
my values.yaml
Copy code
global:
#  logging:
#    level: INFO

  auth:
    cookieSecureSetting: "false"

  database:
    type: external
    host: "{{ airbyte_database_host|lower }}"
    port: "{{ airbyte_database_port }}"
    database: "{{ airbyte_database }}"
    user: "{{ airbyte_database_user }}"
    secretName: "airbyte-config-secrets"
    passwordSecretKey: "database-password"

worker:
  maxNotifyWorkers: 1

postgresql:
  enabled: false
  
temporal:
  externalSSL: false
  extraEnv:
    - name: SKIP_DB_CREATE
      value: "true"
    - name: POSTGRES_SEEDS
      value: "{{ airbyte_database_host|lower }}"
    - name: DB_PORT
      value: "{{ airbyte_database_port }}"
    - name: POSTGRES_USER
      value: "userairbyte"
    - name: POSTGRES_PWD
      value: "userairbyte"
    - name: POSTGRES_TLS_ENABLED
      value: "false"
    - name: POSTGRES_TLS_DISABLE_HOST_VERIFICATION
      value: "true"
    - name: SQL_TLS_ENABLED
      value: "false"
    - name: SQL_TLS_DISABLE_HOST_VERIFICATION
      value: "true"
WARNING Encountered an issue deploying Airbyte: Pod: airbyte-abctl-server-67c74667bc-vlqr9.1899b082f26ca7c6 Reason: Unhealthy Message: Readiness probe failed: Get "http://10.244.0.12:8001/api/v1/health": dial tcp 10.244.0.128001 connect: connection refused Count: 6
k
The knowledge sources don’t give a single “this exact error means X” answer for the 2.0.x 
server
readiness probe failing with 
connection refused
, but they do show that: • 
abctl
will report exactly this kind of message when the 
airbyte-abctl-server
pod is not becoming ready. [Issue 50430] • The generic guidance is: when 
abctl
says “Deployment is not ready” or a pod is 
Unhealthy
, you must inspect that pod’s logs to see the real cause. [Troubleshoot abctl] Given your command and 
values.yaml
, there are two likely problem areas: 1. External DB config for chart V2 For Helm chart V2, the external DB block should use 
name:
(not 
database:
) for the DB name. The docs show both V1 and V2 syntaxes; for V2 it’s:
Copy code
postgresql:
     enabled: false

   global:
     database:
       type: external
       secretName: "airbyte-config-secrets"
       host: "<your-db-host>"
       port: "<your-db-port>"
       name: "<your-db-name>"          # <-- V2 key
       userSecretKey: "database-user"
       passwordSecretKey: "database-password"
[External DB 2.0] Your 
values.yaml
uses 
database:
and 
user:
directly under 
global.database
, which matches the V1 example, not the V2 one. [External DB (V1 vs V2); External DB 2.0] That mismatch can cause the server to fail on startup (e.g., can’t read DB config), which then surfaces only as a readiness probe 
connection refused
. Action (based on docs): Adjust your DB block to the V2 form, for example:
Copy code
postgresql:
     enabled: false

   global:
     auth:
       cookieSecureSetting: "false"

     database:
       type: external
       secretName: "airbyte-config-secrets"

       host: "{{ airbyte_database_host|lower }}"
       port: "{{ airbyte_database_port }}"
       name: "{{ airbyte_database }}"          # was `database`

       userSecretKey: "database-user"          # use secret key instead of inline user
       passwordSecretKey: "database-password"
And ensure your 
airbyte-config-secrets
secret has at least:
Copy code
stringData:
     database-user: <user>
     database-password: <password>
[External DB 2.0] 2. Temporal extra env duplication Your 
temporal.extraEnv
block hard‑codes 
POSTGRES_*
and 
SQL_TLS_*
values. The docs don’t show this pattern for external DB with chart V2; instead, they rely on the 
global.database
config and secrets. [External DB 2.0] In earlier errors you saw, duplicated/contradictory env vars caused Helm patch failures. [Patch order error] While your current error is at the server readiness level, it’s safer (per docs) to avoid overriding core DB env vars unless you have a specific, documented need. Action (conservative, doc‑aligned): Temporarily remove the 
temporal.extraEnv
overrides and rely on the default chart behavior with 
global.database
configured. ------------------------- Because the readiness probe error is generic, the docs say the next step is to inspect the server pod logs to see the real cause. [Troubleshoot abctl] After updating 
values.yaml
as above, re‑run:
Copy code
abctl local uninstall
abctl local install --host HOST \
  --insecure-cookies --no-browser \
  --chart-version 2.0.19 \
  --secret /secret.yaml \
  --values /values.yaml \
  --docker-username USERNAME --docker-password PASSWORD
If it still fails, get the server logs:
Copy code
kubectl -n airbyte-abctl logs deploy/airbyte-abctl-server
and look for explicit DB or config errors. The knowledge sources don’t include those logs for 2.0.19, so I can’t be more specific without seeing them, but aligning your 
global.database
block with the documented V2 syntax is the main concrete correction the docs support. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai if I dont give the temporal/extraVars, temporal fails connecting. youre right about the global.database.name, my bad but I still have the issue
k
You’re right to keep the 
temporal.extraEnv
if Temporal can’t connect without it. The remaining problem is that the server pod is still failing its readiness probe (
connection refused
), which is generic. The docs say that in this situation you must look at the server logs to see the real cause. [Troubleshoot abctl] At this point, the knowledge sources don’t contain enough information to say exactly why your 
airbyte-abctl-server
is refusing connections after fixing 
global.database.name
. The next step is: 1. Get the server logs:
Copy code
kubectl -n airbyte-abctl logs deploy/airbyte-abctl-server
(or the specific pod if needed):
Copy code
kubectl -n airbyte-abctl get pods | grep server
kubectl -n airbyte-abctl logs <server-pod-name>
2. Look for (and ideally paste here) any stack traces or clear errors, especially around: ◦ Database connection (Hikari / Postgres) ◦ Config loading / secrets ◦ Workload API / auth Without those logs, the docs only give general guidance (inspect bootloader/server logs; most failures are DB or config related) and don’t document a specific 2.0.19 failure mode that matches “server readiness probe connection refused” after fixing 
global.database.name
. [Troubleshoot abctl] If you can share the relevant error lines from 
airbyte-abctl-server
logs, I can try to match them to the documented patterns; otherwise I can’t confidently say more based on the current knowledge sources. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
g
@kapa.ai here is the issue
Copy code
2026-03-04 16:44:25,381 [main]	INFO	i.t.s.WorkflowServiceStubsImpl(<init>):86 - Created WorkflowServiceStubs for channel: ManagedChannelOrphanWrapper{delegate=ManagedChannelImpl{logId=1, target=airbyte-abctl-temporal:7233}}
2026-03-04 16:44:25,879 [main]	WARN	i.a.c.t.TemporalUtils(getTemporalClientWhenConnected):184 - Ignoring exception while trying to request Temporal namespace:
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:351)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:332)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:174)
	at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.getSystemInfo(WorkflowServiceGrpc.java:6065)
	at io.temporal.serviceclient.SystemInfoInterceptor.getServerCapabilitiesOrThrow(SystemInfoInterceptor.java:132)
	at io.temporal.serviceclient.SystemInfoInterceptor$1.start(SystemInfoInterceptor.java:86)
	at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:418)
	at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:394)
	at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:307)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:163)
	at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.describeNamespace(WorkflowServiceGrpc.java:5531)
	at io.airbyte.commons.temporal.TemporalUtils.getNamespaceInfo(TemporalUtils.kt:202)
	at io.airbyte.commons.temporal.TemporalUtils.getTemporalClientWhenConnected(TemporalUtils.kt:180)
	at io.airbyte.commons.temporal.TemporalUtils.createTemporalService(TemporalUtils.kt:77)
	at io.airbyte.commons.temporal.TemporalUtils.createTemporalService(TemporalUtils.kt:103)
	at io.airbyte.commons.temporal.config.TemporalBeanFactory.temporalService(TemporalBeanFactory.kt:30)
	at io.airbyte.commons.temporal.config.$TemporalBeanFactory$TemporalService0$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.commons.temporal.config.$TemporalBeanFactory$WorkflowClient2$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.commons.temporal.config.$TemporalBeanFactory$WorkflowClientWrapped3$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.commons.temporal.$TemporalClient$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.server.config.$TemporalBeanFactory$SynchronousSchedulerClient0$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.commons.server.handlers.helpers.$ActorDefinitionHandlerHelper$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.commons.server.handlers.$SourceDefinitionsHandler$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:3000)
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2762)
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1781)
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:210)
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2130)
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1360)
	at io.airbyte.server.apis.controllers.$SourceDefinitionApiController$Definition.instantiate(Unknown Source)
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2337)
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3150)
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
	at io.micronaut.context.DefaultBeanContext.intializeEagerBean(DefaultBeanContext.java:3039)
	at io.micronaut.context.DefaultBeanContext.initializeEagerBean(DefaultBeanContext.java:2708)
	at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:2034)
	at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:323)
	at io.micronaut.context.DefaultBeanContext.configureAndStartContext(DefaultBeanContext.java:3350)
	at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:353)
	at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:225)
	at io.micronaut.runtime.Micronaut.start(Micronaut.java:75)
	at io.airbyte.server.ApplicationKt.main(Application.kt:17)
Caused by: io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: airbyte-abctl-temporal/10.96.243.202:7233
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused
	at io.grpc.netty.shaded.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166)
	at io.grpc.netty.shaded.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131)
	at io.grpc.netty.shaded.io.netty.channel.unix.Socket.finishConnect(Socket.java:359)
	at io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:715)
	at io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:692)
	at io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567)
	at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:491)
	at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
	at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
	at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:1583)
k
The logs show that the Airbyte server cannot connect to the Temporal service: > 
finishConnect(..) failed: Connection refused: airbyte-abctl-temporal/10.96.243.202:7233
This is the same pattern as the open issue where 
airbyte-server
fails during Helm install because it can’t reach Temporal (
Connection refused
on temporal-service`:7233`). [Server–Temporal error] So the problem is not the DB anymore; it’s that Temporal itself is not up/ready or not reachable at 
airbyte-abctl-temporal:7233
. From the knowledge sources, the concrete checks you can do: 1. Check Temporal pod and service status
Copy code
kubectl -n airbyte-abctl get pods | grep temporal
   kubectl -n airbyte-abctl get svc  | grep temporal
◦ If the Temporal pod is 
CrashLoopBackOff
or 
Error
, you need its logs:
Copy code
kubectl -n airbyte-abctl logs deploy/airbyte-abctl-temporal
◦ If the service is missing or has no endpoints, the server won’t be able to connect. The same 
Connection refused
stack trace appears when Temporal is not actually listening on 7233. [Server–Temporal error; Temporal connect refused] 2. Verify Temporal DB connectivity / SSL In a similar case with external Postgres on Azure, Temporal failed because it couldn’t connect to the DB until SSL was configured correctly. [Azure Temporal DB] Since you’re using an external DB and overriding Temporal env vars, double‑check: ◦ 
POSTGRES_SEEDS
and 
DB_PORT
point to the correct host/port. ◦ 
POSTGRES_USER
/ 
POSTGRES_PWD
match a user that can access the Temporal DB(s). ◦ If your Postgres requires SSL, you may need 
POSTGRES_TLS_ENABLED=true
and proper certs; in the Azure case, adding the SSL cert fixed Temporal. [Azure Temporal DB] 3. Be aware of Temporal + networking quirks There is also an issue where Temporal fails to bind/connect correctly in some Kubernetes networking setups (dual‑stack GKE), leading to the same 
Connection refused
on the Temporal service IP/port. [Temporal bind issue] The knowledge sources don’t provide a generic fix for that, only the symptom. ------------------------- The key point from the stack trace is: Airbyte server is failing startup because it cannot reach Temporal at 
airbyte-abctl-temporal:7233
. [Server–Temporal error] To move forward, you’ll need to: 1. Confirm that the Temporal pod is running and ready. 2. If it isn’t, inspect the Temporal pod logs to see why it can’t start (likely DB/SSL or env misconfig). 3. Fix Temporal so it can start and listen on 7233; once Temporal is healthy, the server readiness probe should pass. The knowledge sources do not include your Temporal pod logs, so I can’t say exactly why Temporal is failing in your environment, only that the server error is a direct consequence of Temporal not being reachable. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
7 Views
Open in Slack
PreviousNext
Powered by