https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> I get `2025-12-22 14:58:53,520 [mai...
# replication-ask-ai
i
@kapa.ai I get 
2025-12-22 14:58:53,520 [main]	ERROR	i.m.r.Micronaut(handleStartupException):362 - Error starting Micronaut server: Failed to initialize data-plane
java.lang.RuntimeException: Failed to initialize data-plane
On mu workload launcher, I have the airbyte-config-secrets and I have botht the dataplane client id an d secret in there, it's the ones he created first, I don't know how to make it work now ?
k
The knowledge sources show this exact error (
Failed to initialize data-plane
) happening when the workload launcher cannot correctly read or use the dataplane credentials from the Kubernetes secret, even if the keys seem to be present. [[Multiple issues 2.0.0](https://github.com/airbytehq/airbyte/issues/68134)] However, they do not provide a definitive, step‑by‑step fix for your specific situation, so I can only outline what is documented and what has helped others. Key points from the docs and issues: 1. Secret name and keys must match what the chart expects For Enterprise Flex / data-plane deployments, the Helm values must reference the same secret name and keys you actually created:
Copy code
yaml    dataPlane:      secretName: airbyte-config-secrets      id: "preview-data-plane"       clientIdSecretName: airbyte-config-secrets      clientIdSecretKey: DATA_PLANE_CLIENT_ID      clientSecretSecretName: airbyte-config-secrets      clientSecretSecretKey: DATA_PLANE_CLIENT_SECRET
And the secret should look like:
Copy code
yaml    apiVersion: v1    kind: Secret    metadata:      name: airbyte-config-secrets    type: Opaque    stringData:      DATA_PLANE_CLIENT_ID: your-data-plane-client-id      DATA_PLANE_CLIENT_SECRET: your-data-plane-client-secret      # optional, only if also set on control plane:      AWS_SECRET_MANAGER_ACCESS_KEY_ID: ...      AWS_SECRET_MANAGER_SECRET_ACCESS_KEY: ...      S3_ACCESS_KEY_ID: ...      S3_SECRET_ACCESS_KEY: ...
[[Flex data-plane 2.0](https://docs.airbyte.com/platform/2.0/enterprise-flex/data-plane#step-3); [Flex data-plane](https://docs.airbyte.com/platform/enterprise-flex/data-plane#step-3)] Please double‑check: - The secret name used by the workload launcher deployment matches 
dataPlane.secretName
. - The keys are exactly 
DATA_PLANE_CLIENT_ID
and 
DATA_PLANE_CLIENT_SECRET
(case‑sensitive). 2. Secret manager configuration must match control plane If you are using a secrets manager (AWS, Azure, etc.), the data plane’s 
secretsManager
section must match the control plane configuration. [[Flex data-plane](https://docs.airbyte.com/platform/enterprise-flex/data-plane#step-3)] If this is misconfigured, the workload launcher can fail during data‑plane initialization even if the Kubernetes secret exists. 3. Pre‑created / externally managed secrets can cause issues Several users report that when the auth / dataplane secrets are pre‑created and managed externally, the bootloader / workload launcher may fail to initialize or may not update the secret as expected. In some cases, deleting the secret and letting Airbyte recreate it resolved dataplane initialization problems. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] The knowledge sources do not give a guaranteed procedure for your exact version, but they suggest: - Ensure the secret is not blocked from being updated by Airbyte. - If possible in your environment, delete the secret and redeploy so Airbyte can recreate it, then verify the new values. 4. Version‑specific behavior There were known issues in 1.6.x and early 1.8.x around dataplane credentials and secrets (e.g., bootloader rewriting secrets, missing keys, etc.). [[1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468); [bootloader auth secret behavior](https://github.com/airbytehq/airbyte/issues/64159)] The fix mentioned there (checking existing credentials in the secret before rewriting) is available from 1.8.0, but the sources don’t describe behavior for 2.x in detail. --- Because the sources don’t include a precise fix for your exact error and setup, I can’t say with certainty what will resolve it. Based on what is documented, the most concrete checks you can do now are: 1. Confirm the workload launcher pod’s environment actually has the expected variables: - 
DATA_PLANE_CLIENT_ID
- 
DATA_PLANE_CLIENT_SECRET
2. Confirm the Helm values (
dataPlane.*
and 
secretsManager.*
) exactly match your secret name and keys and your control plane’s secrets manager configuration. 3. If you are comfortable doing so, delete and recreate 
airbyte-config-secrets
using the documented manifest, then redeploy the workload launcher. If you can share: - Your Airbyte version - The relevant part of your 
values.yaml
(dataPlane + secretsManager) - The exact YAML of your 
airbyte-config-secrets
(with values redacted) I can compare them against the docs and help you pinpoint any mismatch.
i
@kapa.ai - name: DATAPLANE_CLIENT_ID valueFrom: secretKeyRef: key: dataplane-client-id name: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_NAME value: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_KEY value: dataplane-client-id - name: DATAPLANE_CLIENT_SECRET valueFrom: secretKeyRef: key: dataplane-client-secret name: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_NAME value: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_KEY value: dataplane-client-secret this is the configuration for workload launcher
k
A message from kapa.ai
i
@kapa.ai apiVersion: v1 data: dataplane-client-id: NTAxMzAyMDgtOGY3MS00MDk3LThiMzQtOWEzMDk5OTJhOWUz dataplane-client-secret: ZmYwZmU1YWU3ZTdlNzA2NTNiNWZlZWVkMThkOWY5YjAzMjZlNTIwZDVjNTIzYmQ4NGNiMDZlM2ZkNWIwOWNmZQ== instance-admin-client-id: NzEwZjg1YWItNjA2YS00ZDQ3LWJkYzQtNjhhNTc4ZjEyNmE1 instance-admin-client-secret: ZU9WaXFqQWprWkw0UDNIOXRTUU1jRTZEOXpXdU5mU0U= instance-admin-password: bFVnWlZtZ2hqSjVpcGNueG04c09YTmxLWkd0SGJyNFA= jwt-signature-secret: OFp4QzRQOUI3V0pyWVYwTUBRa0g2RDJzRjVMM0E= kind: Secret metadata: annotations: reconcile.external-secrets.io/data-hash: 225c33e5ac65892a97800b745a9849682f3aeb9c24c1e33859416e7f creationTimestamp: "2025-12-19T101924Z" labels: reconcile.external-secrets.io/created-by: 5d2e7a57d4cd4606136c15ded749f33d2fd0a530610b029b3aab2854 reconcile.external-secrets.io/managed: "true" name: airbyte-config-secrets namespace: airbyte ownerReferences: - apiVersion: external-secrets.io/v1 blockOwnerDeletion: true controller: true kind: ExternalSecret name: airbyte-config-secrets uid: 62d4b000-1b04-4d4a-9986-305bef7d58d1 resourceVersion: "686496865" uid: 587fd4b2-42e5-460d-8b11-cf64be2f9dc8 type: Opaque and this is my kubernetes secert
k
The knowledge sources don’t describe this exact env‑var pattern (
dataplane-client-id
/ 
dataplane-client-secret
keys in the secret) in detail, so I can’t be fully certain, but there is a likely mismatch between what your chart is using and what the current docs expect. From the docs for data planes (Enterprise / Flex), the Kubernetes secret keys are expected to be:
Copy code
yaml stringData:   DATA_PLANE_CLIENT_ID: your-data-plane-client-id   DATA_PLANE_CLIENT_SECRET: your-data-plane-client-secret
and the Helm values then point to those keys:
Copy code
yaml dataPlane:   secretName: airbyte-config-secrets   id: "preview-data-plane"    clientIdSecretName: airbyte-config-secrets   clientIdSecretKey: DATA_PLANE_CLIENT_ID   clientSecretSecretName: airbyte-config-secrets   clientSecretSecretKey: DATA_PLANE_CLIENT_SECRET
[[Flex data-plane](https://docs.airbyte.com/platform/enterprise-flex/data-plane#step-3); [Flex data-plane 2.0](https://docs.airbyte.com/platform/2.0/enterprise-flex/data-plane#step-3)] In your workload launcher, the env vars are:
Copy code
yaml - name: DATAPLANE_CLIENT_ID   valueFrom:     secretKeyRef:       key: dataplane-client-id       name: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_NAME   value: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_KEY   value: dataplane-client-id  - name: DATAPLANE_CLIENT_SECRET   valueFrom:     secretKeyRef:       key: dataplane-client-secret       name: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_NAME   value: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_KEY   value: dataplane-client-secret
So the chart is looking for lower‑case keys 
dataplane-client-id
and 
dataplane-client-secret
in the secret, while the docs show upper‑case 
DATA_PLANE_CLIENT_ID
and 
DATA_PLANE_CLIENT_SECRET
. [[Enterprise multi‑region](https://docs.airbyte.com/platform/enterprise-setup/multi-region#step-4)] Given that: 1. Check your 
airbyte-config-secrets
content: - If it has 
DATA_PLANE_CLIENT_ID
/ 
DATA_PLANE_CLIENT_SECRET
(upper‑case, with underscores), but the pod is using 
dataplane-client-id
/ 
dataplane-client-secret
, the workload launcher will not find the values and can fail with 
Failed to initialize data-plane
. - Conversely, if your secret only has the lower‑case keys but your Helm 
values.yaml
is configured per docs (pointing to 
DATA_PLANE_CLIENT_ID
/ 
DATA_PLANE_CLIENT_SECRET
), there’s also a mismatch. 2. Align them one way or the other: - Easiest path following docs: define the secret with 
DATA_PLANE_CLIENT_ID
and 
DATA_PLANE_CLIENT_SECRET
and set in `values.yaml`:
Copy code
yaml      dataPlane:        clientIdSecretName: airbyte-config-secrets        clientIdSecretKey: DATA_PLANE_CLIENT_ID        clientSecretSecretName: airbyte-config-secrets        clientSecretSecretKey: DATA_PLANE_CLIENT_SECRET
- Then ensure the rendered workload launcher env uses those same keys (if it doesn’t, you may be on a chart version with different expectations). Because the sources don’t include your exact chart version or a mapping between 
dataplane-client-id
and 
DATA_PLANE_CLIENT_ID
, I can’t say which is “correct” for your deployment, only that they must match. If you can share: - The 
airbyte-config-secrets
YAML (keys only, values redacted), and - The 
dataPlane
section of your 
values.yaml
, I can compare them against the documented patterns and point out the exact mismatch.
A message from kapa.ai
There’s not enough information in the knowledge sources to explain this exact failure mode with ExternalSecrets, but they do show how the data‑plane secret is expected to look and how the workload launcher reads it. I’ll stick to what we can infer from that. From your workload launcher:
Copy code
yaml - name: DATAPLANE_CLIENT_ID   valueFrom:     secretKeyRef:       key: dataplane-client-id       name: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_NAME   value: airbyte-config-secrets - name: DATAPLANE_CLIENT_ID_SECRET_KEY   value: dataplane-client-id  - name: DATAPLANE_CLIENT_SECRET   valueFrom:     secretKeyRef:       key: dataplane-client-secret       name: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_NAME   value: airbyte-config-secrets - name: DATAPLANE_CLIENT_SECRET_SECRET_KEY   value: dataplane-client-secret
And your `airbyte-config-secrets`:
Copy code
yaml apiVersion: v1 kind: Secret metadata:   name: airbyte-config-secrets   namespace: airbyte type: Opaque data:   dataplane-client-id: ...   dataplane-client-secret: ...   instance-admin-client-id: ...   instance-admin-client-secret: ...   instance-admin-password: ...   jwt-signature-secret: ...
So: - The secret name matches (
airbyte-config-secrets
). - The keys 
dataplane-client-id
and 
dataplane-client-secret
exist and are base64‑encoded, as required for 
data:
. This means the simple “key not found” problem (seen in earlier 1.6.x issues) is not what you’re hitting. [[1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468)] From the issues, two important behaviors are mentioned: 1. The bootloader / workload launcher can rewrite dataplane credentials in the auth/config secret, even if you pre‑populate them. [[1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468); [Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] 2. Users who had externally managed secrets sometimes had to delete the secret and let Airbyte recreate it so that the internal state (database table 
dataplane_client_credentials
) and the secret contents matched. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] In your case, the secret is clearly managed by ExternalSecrets (annotations and ownerReferences). That likely prevents Airbyte from updating it if it wants to rotate or align the dataplane credentials, which can lead to a mismatch between: - What’s in 
airbyte-config-secrets
(your ExternalSecret‑managed values), and - What Airbyte expects internally (what it has in its 
dataplane_client_credentials
table). When that mismatch happens, the bootloader / workload launcher can fail with a generic 
Failed to initialize data-plane
error, even though the env vars are present. Given what the sources say, the most concrete, doc‑aligned steps you can try are: 1. Temporarily remove ExternalSecrets control for this secret - Stop reconciling 
airbyte-config-secrets
via ExternalSecret (or create a separate secret for ExternalSecrets and let Airbyte own 
airbyte-config-secrets
). - Delete the existing `airbyte-config-secrets`:
Copy code
bash      kubectl delete secret airbyte-config-secrets -n airbyte
- Redeploy / restart Airbyte so it can recreate and populate the secret itself. This is exactly what one user did to get the dataplane client id/secret created correctly and make the workload launcher start. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] 2. If you must keep ExternalSecrets, align it with what Airbyte expects: - First, let Airbyte create/update 
airbyte-config-secrets
once (as above), then: - Read back the resulting keys/values from the secret. - Configure your ExternalSecret to mirror those exact keys and values going forward. - Ensure ExternalSecrets is not constantly overwriting values that Airbyte rotates. 3. Double‑check that the dataplane credentials are valid against the control plane In the 1.6.x issue, a user validated the dataplane client id/secret by calling the 
/api/v1/dataplanes/token
endpoint and confirming they got an access token. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] If your control plane exposes a similar endpoint, you can: - Base64‑decode 
dataplane-client-id
and 
dataplane-client-secret
from the secret. - Use them in a token request to confirm they’re accepted. Because the knowledge sources don’t document ExternalSecrets integration or your exact version, I can’t guarantee this will fix it, but based on the issues, the most likely root cause is: > Airbyte expects to control and/or synchronize the dataplane credentials in 
airbyte-config-secrets
, but ExternalSecrets is managing that secret instead, causing a mismatch and 
Failed to initialize data-plane
. If you can share: - Your Airbyte version and Helm chart version, and - Whether you’re on Enterprise Flex / multi‑region / OSS, I can cross‑check against the relevant docs section (multi‑region vs Flex) and see if there’s any version‑specific nuance mentioned.
i
@kapa.ai I also can't get to the UI becaus eof this I keep getting this errors 2025-12-22 145759,960 [io-executor-thread-7] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 145759,963 [io-executor-thread-7] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Could not generate token at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken$lambda$0(DataplaneTokenServiceDataImpl.kt:53) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken(DataplaneTokenServiceDataImpl.kt:52) at io.airbyte.server.services.DataplaneService.getToken(DataplaneService.kt:57) at io.airbyte.server.apis.controllers.DataplaneController.getDataplaneToken(DataplaneController.kt:134) at io.airbyte.server.apis.controllers.$DataplaneController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 2025-12-22 145811,974 [io-executor-thread-8] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 145852,974 [io-executor-thread-9] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 145852,976 [io-executor-thread-9] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Could not generate token at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken$lambda$0(DataplaneTokenServiceDataImpl.kt:53) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken(DataplaneTokenServiceDataImpl.kt:52) at io.airbyte.server.services.DataplaneService.getToken(DataplaneService.kt:57) at io.airbyte.server.apis.controllers.DataplaneController.getDataplaneToken(DataplaneController.kt:134) at io.airbyte.server.apis.controllers.$DataplaneController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 2025-12-22 145853,508 [io-executor-thread-10] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 145853,510 [io-executor-thread-10] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Could not generate token at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken$lambda$0(DataplaneTokenServiceDataImpl.kt:53) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken(DataplaneTokenServiceDataImpl.kt:52) at io.airbyte.server.services.DataplaneService.getToken(DataplaneService.kt:57) at io.airbyte.server.apis.controllers.DataplaneController.getDataplaneToken(DataplaneController.kt:134) at io.airbyte.server.apis.controllers.$DataplaneController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 2025-12-22 150029,470 [io-executor-thread-1] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 150029,476 [io-executor-thread-1] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Could not generate token at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken$lambda$0(DataplaneTokenServiceDataImpl.kt:53) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken(DataplaneTokenServiceDataImpl.kt:52) at io.airbyte.server.services.DataplaneService.getToken(DataplaneService.kt:57) at io.airbyte.server.apis.controllers.DataplaneController.getDataplaneToken(DataplaneController.kt:134) at io.airbyte.server.apis.controllers.$DataplaneController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 2025-12-22 150030,906 [io-executor-thread-2] WARN i.m.s.t.j.g.JwtTokenGenerator(generateToken):151 - JOSEException while generating token The secret length must be at least 256 bits 2025-12-22 150030,909 [io-executor-thread-2] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Could not generate token at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken$lambda$0(DataplaneTokenServiceDataImpl.kt:53) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at io.airbyte.data.services.impls.data.DataplaneTokenServiceDataImpl.getToken(DataplaneTokenServiceDataImpl.kt:52) at io.airbyte.server.services.DataplaneService.getToken(DataplaneService.kt:57) at
k
A message from kapa.ai
The knowledge sources don’t mention this specific JWT error, so I can’t be fully certain, but they do show how Airbyte expects secrets to be shaped and that misconfigured / externally‑managed secrets can break dataplane initialization and token generation. [[Enterprise multi‑region](https://docs.airbyte.com/platform/enterprise-setup/multi-region#step-4); [Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] From your logs: > 
JOSEException while generating token The secret length must be at least 256 bits
> 
IllegalStateException: Could not generate token
This is coming from the JWT token generator. That usually means the JWT signing secret it reads from Kubernetes is too short or invalid. In your 
airbyte-config-secrets
you have:
Copy code
yaml data:   ...   jwt-signature-secret: OFp4QzRQOUI3V0pyWVYwTUBRa0g2RDJzRjVMM0E=
Key points: 1. JWT secret must be long enough The error explicitly says the secret must be at least 256 bits (32 bytes). If the underlying decoded value of 
jwt-signature-secret
is shorter than that, token generation will fail and the UI cannot authenticate. The docs don’t give an exact length requirement, but they do show that sensitive values (license key, DB password, etc.) are stored as strings in 
stringData
or base64 in 
data
. [[Creating secrets](https://docs.airbyte.com/platform/deploying-airbyte/creating-secrets); [Enterprise implementation guide](https://docs.airbyte.com/platform/enterprise-setup/implementation-guide#creating-a-kubernetes-secret)] In your case, 
jwt-signature-secret
is in 
data:
, so it is base64‑encoded. You need to ensure that the decoded value is at least 32 bytes and suitably random. 2. Secret is externally managed Your secret is owned by an `ExternalSecret`:
Copy code
yaml    annotations:      <http://reconcile.external-secrets.io/managed|reconcile.external-secrets.io/managed>: "true"    ownerReferences:      kind: ExternalSecret
The issues show that when Airbyte needs to create or update secrets (including dataplane credentials), having them managed externally can cause mismatches and failures. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587); [1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468)] It’s likely the same here: Airbyte expects a valid JWT signing secret, but ExternalSecrets is providing one that doesn’t meet the length requirement, and Airbyte cannot change it. 3. What you can do Based on the docs and issues, the most concrete, safe steps are: 1. Fix 
jwt-signature-secret
length at the source (ExternalSecret) - Generate a new random secret of at least 32 bytes (e.g. 32+ random characters). - Base64‑encode it and store that in your external secret backend so that Kubernetes 
data.jwt-signature-secret
is the base64 of that value. - Let ExternalSecrets reconcile and update 
airbyte-config-secrets
. 2. Restart the control plane / server pods After the secret is updated, restart the Airbyte server (and workload launcher if needed) so they pick up the new JWT secret. 3. If possible, temporarily let Airbyte own the secret As with dataplane credentials, one documented workaround is to: - Stop ExternalSecrets from managing 
airbyte-config-secrets
. - Delete the secret:
Copy code
bash         kubectl delete secret airbyte-config-secrets -n airbyte
- Redeploy Airbyte so it can recreate/populate the secret itself. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] Then, once everything works (UI accessible, dataplane initialized), you can mirror those values into your external secret system if you still want external management. Because the docs don’t show the exact variable name for the JWT secret or its required length, I can’t guarantee this is the only issue, but: - Your dataplane client id/secret keys match what the workload launcher expects. - The error stack is clearly about JWT token generation, not missing dataplane env vars. - You have a 
jwt-signature-secret
that may not meet the 256‑bit requirement and is controlled by ExternalSecrets. So the most likely fix is: update 
jwt-signature-secret
(via your ExternalSecret) to a sufficiently long, random value, then restart Airbyte. If you can share: - How you define the ExternalSecret for 
airbyte-config-secrets
(YAML, with values redacted), and - Your Airbyte version / Helm chart version, I can compare that against the relevant docs section (Enterprise / Flex / multi‑region) and see if there are any other required keys you might be missing.
i
@kapa.ai generallty spoeaking 
2025-12-22 15:10:22,800 [scheduled-executor-thread-2]	ERROR	i.a.a.c.a.AccessTokenInterceptor(intercept):120 - Failed to obtain or add access token
java.lang.Exception: Token request failed with HTTP code 500
at io.airbyte.api.client.auth.AccessTokenInterceptor.fetchNewToken(AccessTokenInterceptor.kt:98)
at io.airbyte.api.client.auth.AccessTokenInterceptor.getCachedToken(AccessTokenInterceptor.kt:70)
at io.airbyte.api.client.auth.AccessTokenInterceptor.intercept(AccessTokenInterceptor.kt:116)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at io.airbyte.api.client.interceptor.ThrowOn5xxInterceptor.intercept(ThrowOn5xxInterceptor.kt:14)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
at dev.failsafe.okhttp.FailsafeCall.lambda$execute$0(FailsafeCall.java:117)
at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
at dev.failsafe.CallImpl.execute(CallImpl.java:33)
at dev.failsafe.okhttp.FailsafeCall.execute(FailsafeCall.java:121)
at io.airbyte.api.client.generated.DataplaneApi.heartbeatDataplaneWithHttpInfo(DataplaneApi.kt:1889)
at io.airbyte.api.client.generated.DataplaneApi.heartbeatDataplane(DataplaneApi.kt:292)
at io.airbyte.workload.launcher.authn.DataplaneIdentityService.heartbeat(DataplaneIdentityService.kt:68)
at io.airbyte.workload.launcher.authn.$DataplaneIdentityService$Definition$Exec.dispatch(Unknown Source)
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456)
at io.micronaut.inject.DelegatingExecutableMethod.invoke(DelegatingExecutableMethod.java:86)
at io.micronaut.context.bind.DefaultExecutableBeanContextBinder$ContextBoundExecutable.invoke(DefaultExecutableBeanContextBinder.java:152)
at io.micronaut.scheduling.processor.ScheduledMethodProcessor.lambda$scheduleTask$2(ScheduledMethodProcessor.java:160)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
2025-12-22 15:10:22,920 [scheduled-executor-thread-2]	WARN	i.a.w.l.a.DataplaneIdentityService(heartbeat):82 - Failed to authenticate, stopping.
2025-12-22 15:10:23,052 [main]	INFO	i.a.f.ConfigFileClient(<init>):150 - path /flags does not exist, will return default flag values
2025-12-22 15:10:23,420 [main]	ERROR	i.a.a.c.a.AccessTokenInterceptor(intercept):120 - Failed to obtain or add access token
java.lang.Exception: Token request failed with HTTP code 500
at io.airbyte.api.client.auth.AccessTokenInterceptor.fetchNewToken(AccessTokenInterceptor.kt:98)
at io.airbyte.api.client.auth.AccessTokenInterceptor.getCachedToken(AccessTokenInterceptor.kt:70)
at io.airbyte.api.client.auth.AccessTokenInterceptor.intercept(AccessTokenInterceptor.kt:116)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at io.airbyte.api.client.interceptor.ThrowOn5xxInterceptor.intercept(ThrowOn5xxInterceptor.kt:14)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
at dev.failsafe.okhttp.FailsafeCall.lambda$execute$0(FailsafeCall.java:117)
at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
at dev.failsafe.CallImpl.execute(CallImpl.java:33)
at dev.failsafe.okhttp.FailsafeCall.execute(FailsafeCall.java:121)
at io.airbyte.api.client.generated.DataplaneApi.initializeDataplaneWithHttpInfo(DataplaneApi.kt:2244)
at io.airbyte.api.client.generated.DataplaneApi.initializeDataplane(DataplaneApi.kt:364)
at io.airbyte.workload.launcher.authn.DataplaneIdentityService.initialize(DataplaneIdentityService.kt:41)
at io.airbyte.workload.launcher.StartupApplicationEventListener.onApplicationEvent(StartupApplicationEventListener.kt:34)
at io.airbyte.workload.launcher.StartupApplicationEventListener.onApplicationEvent(StartupApplicationEventListener.kt:20)
at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:266)
at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226)
at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery.onStart(NettyServiceDiscovery.java:54)
at io.micronaut.http.server.netty.discovery.$NettyServiceDiscovery$Definition$Exec.dispatch(Unknown Source)
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456)
at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:3987)
at io.micronaut.aop.chain.AdapterIntroduction.intercept(AdapterIntroduction.java:91)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143)
at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery$ApplicationEventListener$onStart1$Intercepted.onApplicationEvent(Unknown Source)
at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:266)
at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226)
at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:720)
at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:330)
at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:124)
at io.micronaut.runtime.Micronaut.start(Micronaut.java:81)
at io.airbyte.workload.launcher.ApplicationKt.main(Application.kt:10)
2025-12-22 15:10:23,433 [main]	ERROR	i.m.r.Micronaut(handleStartupException):362 - Error starting Micronaut server: Failed to initialize data-plane
java.lang.RuntimeException: Failed to initialize data-plane
at io.airbyte.workload.launcher.authn.DataplaneIdentityService.initialize(DataplaneIdentityService.kt:56)
at io.airbyte.workload.launcher.StartupApplicationEventListener.onApplicationEvent(StartupApplicationEventListener.kt:34)
at io.airbyte.workload.launcher.StartupApplicationEventListener.onApplicationEvent(StartupApplicationEventListener.kt:20)
at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:266)
at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226)
at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery.onStart(NettyServiceDiscovery.java:54)
at io.micronaut.http.server.netty.discovery.$NettyServiceDiscovery$Definition$Exec.dispatch(Unknown Source)
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456)
at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:3987)
at io.micronaut.aop.chain.AdapterIntroduction.intercept(AdapterIntroduction.java:91)
at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143)
at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery$ApplicationEventListener$onStart1$Intercepted.onApplicationEvent(Unknown Source)
at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:266)
at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226)
at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:720)
at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:330)
at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:124)
at io.micronaut.runtime.Micronaut.start(Micronaut.java:81)
at io.airbyte.workload.launcher.ApplicationKt.main(Application.kt:10)
Caused by: org.openapitools.client.infrastructure.ClientException: Client error : 401 Unauthorized
at io.airbyte.api.client.generated.DataplaneApi.initializeDataplane(DataplaneApi.kt:372)
at io.airbyte.workload.launcher.authn.DataplaneIdentityService.initialize(DataplaneIdentityService.kt:41)
... 18 common frames omitted
what doe sthis airbyte issue say ?
k
A message from kapa.ai
The knowledge sources don’t contain this exact stack trace, but they do describe very similar dataplane/auth failures, so I can only infer based on those. From your logs: - 
AccessTokenInterceptor
fails with 
HTTP code 500
when calling 
DataplaneApi.initializeDataplane
/ 
heartbeatDataplane
. - The final cause is 
ClientException: 401 Unauthorized
from the control plane when the workload launcher tries to initialize the dataplane. In the issues, this pattern appears when: 1. Dataplane client credentials are wrong or don’t match what the control plane expects, so the control plane rejects the token request (401) and dataplane initialization fails. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587); [1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468)] 2. The secret holding dataplane credentials is pre‑created / externally managed, and Airbyte cannot update it to match its internal 
dataplane_client_credentials
table. Users who: - Deleted the secret, and - Let Airbyte recreate/populate it were able to get dataplane initialization and token requests working again. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] 3. In later versions, Airbyte was changed so that if valid dataplane credentials already exist in the secret, the bootloader skips re‑initializing them. [[bootloader auth secret](https://github.com/airbytehq/airbyte/issues/64159)] Applied to your logs, “generally speaking” this Airbyte error means: > The workload launcher (dataplane) is trying to authenticate against the control plane using the dataplane client id/secret from your Kubernetes secret, but the control plane is returning 401 Unauthorized (and sometimes 500), so the dataplane cannot be initialized and the Micronaut server exits with 
Failed to initialize data-plane
. Given the issues, the likely causes are: - The dataplane client id/secret in 
airbyte-config-secrets
don’t match what the control plane has registered in its 
dataplane_client_credentials
table, or - The control plane’s dataplane token endpoint is misconfigured / not reachable, or - External management of the secret prevents Airbyte from aligning the credentials. The documented remediations in similar cases are: - Ensure the dataplane client id/secret in the secret exactly match the values in the control plane database (or let Airbyte recreate the secret so they are in sync). [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] - Avoid pre‑creating / externally managing the auth/config secret during initial dataplane setup, or at least allow Airbyte to update it once. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587); [1.6.0 dataplane-client-id missing](https://github.com/airbytehq/airbyte/issues/62468)] The sources don’t provide more detailed, version‑specific guidance for your exact stack trace, so I can’t be more precise than that.
i
how can I know what are the credentials of the data plane ? can I find them in sql ?
@kapa.ai how can I know what are the credentials of the data plane ? can I find them in sql ?
k
A message from kapa.ai
The knowledge sources do not explicitly document where dataplane credentials are stored in SQL or how to query them directly, so I can’t give you an exact SQL statement. However, multiple GitHub issues confirm that: - Dataplane client credentials are persisted in the Airbyte database (users refer to a 
dataplane_client_credentials
table and 
dataplane
table). [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] - Some users have inspected that table and manually copied values from there into Kubernetes secrets to work around issues. [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)] So yes, the credentials do exist in SQL, but: - The docs do not provide a supported procedure or schema reference for reading them directly. - The recommended, documented way to manage dataplane credentials is via Kubernetes secrets, not by querying the DB. For Enterprise Flex / data-plane, you’re expected to put them into a secret like:
Copy code
yaml   apiVersion: v1   kind: Secret   metadata:     name: airbyte-config-secrets   type: Opaque   stringData:     DATA_PLANE_CLIENT_ID: your-data-plane-client-id     DATA_PLANE_CLIENT_SECRET: your-data-plane-client-secret
and reference that in 
values.yaml
under 
dataPlane.*
. [[Flex data-plane](https://docs.airbyte.com/platform/enterprise-flex/data-plane#step-3); [Flex data-plane 2.0](https://docs.airbyte.com/platform/2.0/enterprise-flex/data-plane#step-3)] Because the schema and exact column names are not documented in the provided sources, I can’t safely tell you how to query them from SQL. If you need to recover or reset dataplane credentials, the safest approach based on the issues is: 1. Let Airbyte recreate/populate the relevant secret (e.g. by deleting a pre‑created secret and redeploying), then 2. Read the credentials from the Kubernetes secret (base64‑decode the values). [[Could not deploy 1.6](https://github.com/airbytehq/airbyte/issues/58587)]
i
@kapa.ai whats airbyte database called as of airbyte 1.8.3
k
A message from kapa.ai
The knowledge sources don’t state a new or special database name for Airbyte 1.8.3. For Kubernetes/Helm deployments around 1.8, the docs consistently refer to: - The Postgres database name as a configurable value in 
values.yaml
, e.g. 
global.database.database
(Helm v1) or 
global.database.name
(Helm v2). [[External DB 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/integrations/database); [Enterprise DB 1.8](https://docs.airbyte.com/platform/1.8/enterprise-setup/implementation-guide#configuring-the-airbyte-database)] - Example internal DB name 
db-airbyte
when backing up the internal Postgres pod. [[Helm v2 upgrade 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/chart-v2-community#how-to-upgrade)] So for 1.8.3: - There is no fixed, hard‑coded database name documented; it’s set by your deployment configuration (
values.yaml
). - In examples, the internal DB is often called 
db-airbyte
, but that’s just a default/example, not a version‑specific change.
i
@kapa.ai the default, the one if I dont insert anythig how is it called
k
A message from kapa.ai
The knowledge sources provided do not state what the default Postgres database name is when you don’t set anything in 
values.yaml
for Airbyte 1.8.3. They only show that: - The database name is configurable via 
global.database.database
(Helm v1) or 
global.database.name
(Helm v2). [[External DB 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/integrations/database); [Enterprise DB 1.8](https://docs.airbyte.com/platform/1.8/enterprise-setup/implementation-guide#configuring-the-airbyte-database)] - For external databases, you must explicitly set the name; no default is documented. [[External DB 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/integrations/database)] Because the docs don’t specify the built‑in default name for the internal Postgres when nothing is configured, I can’t reliably tell you what it is.
i
@kapa.ai it's 
db-airbyte
k
A message from kapa.ai
Thanks for confirming. The knowledge sources only show 
db-airbyte
as an example name when backing up the internal Postgres database (e.g., 
pg_dump -U airbyte db-airbyte
), but they do not explicitly state that 
db-airbyte
is the guaranteed default for all 1.8.x deployments. [[Helm v2 upgrade 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/chart-v2-community#how-to-upgrade)] So while 
db-airbyte
is very commonly used and appears in the official examples, the docs provided here don’t definitively declare it as the default in every configuration.
i
@kapa.ai 
2025-12-22 15:38:40,373 [Thread-2]	ERROR	i.a.w.l.StartupApplicationEventListener(onApplicationEvent$lambda$1):43 - Failed to retrieve and resume claimed workloads, exiting.
io.airbyte.api.client.ApiException: Unauthorized
at io.airbyte.workload.api.client.WorkloadApiClient.workloadList(WorkloadApiClient.kt:283)
at io.airbyte.workload.launcher.ClaimedProcessor.getWorkloadList$lambda$9(ClaimedProcessor.kt:112)
at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
at io.airbyte.workload.launcher.ClaimedProcessor.getWorkloadList(ClaimedProcessor.kt:110)
at io.airbyte.workload.launcher.ClaimedProcessor.retrieveAndProcess(ClaimedProcessor.kt:57)
at io.airbyte.workload.launcher.StartupApplicationEventListener.onApplicationEvent$lambda$1(StartupApplicationEventListener.kt:39)
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)
what does this mean, from workload launcher
k
A message from kapa.ai
This log means your workload launcher is trying to call the Workload API (
workloadList
) and the request is being rejected with HTTP 401 Unauthorized, so it cannot retrieve/resume claimed workloads and exits. From the knowledge sources, this exact pattern appears when: - The auth token between the worker / workload launcher and the Workload API server is misconfigured or mismatched. - Specifically, the 
WORKLOAD_API_BEARER_TOKEN
(and sometimes 
WORKLOAD_API_AUTH_HEADER
) are not the same on all the relevant deployments. [[502 connector setup](https://github.com/airbytehq/airbyte/issues/65567); [502 worker fix](https://github.com/airbytehq/airbyte/issues/66692)] A user with similar 401 errors on Workload API calls fixed it by: 1. Generating a shared token:
Copy code
bash    TOKEN=$(openssl rand -hex 24)
2. Setting the same env vars on the deployments:
Copy code
bash    kubectl -n airbyte-abctl set env deployment/airbyte-abctl-workload-api-server \      WORKLOAD_API_AUTH_HEADER=Authorization \      WORKLOAD_API_BEARER_TOKEN=$TOKEN     kubectl -n airbyte-abctl set env deployment/airbyte-abctl-worker \      WORKLOAD_API_AUTH_HEADER=Authorization \      WORKLOAD_API_BEARER_TOKEN=$TOKEN
(In another report they also set the same token on the worker and workload API server, and restarted.) [[502 connector setup](https://github.com/airbytehq/airbyte/issues/65567); [502 worker fix](https://github.com/airbytehq/airbyte/issues/66692)] So, for your workload launcher: - The error means: its calls to the Workload API are unauthorized. - The likely fix (per the issues) is to ensure that all components that talk to the Workload API (worker, workload launcher, workload‑api‑server) share the same 
WORKLOAD_API_BEARER_TOKEN
(and 
WORKLOAD_API_AUTH_HEADER
if used), then restart those pods. The sources don’t show your exact deployment layout, so I can’t say which specific deployments you must update, but the underlying cause is a token mismatch leading to 401 from the Workload API.
9 Views
Open in Slack
PreviousNext
Powered by