<@U04S8PLJWMT> I've migrated to Airbyte 2.0.1 usin...
# ask-ai
e
@kapa.ai I've migrated to Airbyte 2.0.1 using helm charts v1, but I now get the following error:
Error: couldn't find key dataplane-client-id in Secret airbyte/airbyte-auth-secrets
. What values should this be set to? Note that I'm using the oss airbyte core version. I was previously on the chart 1.5.x and we didnt have that requirement. dataplane is not cited in any of the migration docs
k
You’re seeing this because newer Helm chart versions introduced required data plane credentials (client id/secret) that are read from the
airbyte-auth-secrets
Secret, and the workload-launcher/init containers fail if those keys are missing. [[1.6.0 issue](https://github.com/airbytehq/airbyte/issues/62468); [1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] However, for OSS single-cluster (“no separate data plane”) there is no documented, user-facing value you’re supposed to provide for
dataplane-client-id
/
dataplane-client-secret
, and the official docs and migration guides you referenced indeed don’t describe any manual setup for that. The existing issues show: - The workload-launcher/init containers expect keys like
dataplane-client-id
and
dataplane-client-secret
in
airbyte-auth-secrets
, and fail with exactly your error when they’re missing. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587); [1.6.0 chart issue](https://github.com/airbytehq/airbyte/issues/62468)] - When users pre‑create
airbyte-auth-secrets
(e.g., just for instance admin credentials), the bootloader may not populate or may overwrite the data plane keys, leading to the failure. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587); [bootloader overwrite report](https://github.com/airbytehq/airbyte/issues/64159)] - A workaround that made things work was not managing
airbyte-auth-secrets
externally: remove your custom secret and any Helm values that point to it, let Airbyte/bootloader create and populate that Secret itself; this also caused the data plane credentials to be created and the workload-launcher to become healthy. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] In your case (OSS, Helm v1, 2.0.1): - The knowledge sources do not document what concrete values to set for
dataplane-client-id
/
dataplane-client-secret
in an OSS-only install, nor a supported way to hand‑craft them. - For Enterprise Flex / separate data plane, the docs clearly say you must use the client id/secret issued by the control plane and store them in a secret (e.g.
airbyte-config-secrets
) under keys like
DATA_PLANE_CLIENT_ID
/
DATA_PLANE_CLIENT_SECRET
, then reference them in values.yaml. [[Enterprise-flex data plane](https://docs.airbyte.com/platform/next/enterprise-flex/data-plane#step-3)] That model doesn’t apply directly to OSS core. Given the above, the only behavior we can rely on from the sources is: 1. If you have a manually managed
airbyte-auth-secrets
, that is likely causing the error. 2. Deleting that secret and the corresponding
values.yaml
references and allowing the bootloader to recreate/populate it has resolved this class of errors for other users. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] Because the knowledge sources do not specify how to correctly hand‑set these values for OSS, I can’t tell you “use X or Y string”; they appear to be generated and stored by the platform itself, and manually setting them is fragile (and in some versions, overwritten by bootloader anyway). [[bootloader overwrite report](https://github.com/airbytehq/airbyte/issues/64159); [62468 comments](https://github.com/airbytehq/airbyte/issues/62468)] So, with what we have in the sources: - There is no documented, stable value you should set for
dataplane-client-id
/
dataplane-client-secret
for OSS Helm v1. - The recommended practical step from existing issues is: - Remove your custom
airbyte-auth-secrets
from the cluster; - Remove or comment out any Helm values that explicitly set
auth.instanceAdmin.secretName
,
emailSecretKey
,
passwordSecretKey
to that Secret so the chart can manage it itself; [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - Re‑deploy so the bootloader creates and populates
airbyte-auth-secrets
, including the data plane keys it expects. If you must keep managing instance admin credentials via your own secret, the knowledge sources do not provide a supported pattern for doing that and providing fixed dataplane credentials in OSS; several users report conflicts and inconsistent behavior in exactly that setup. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587); [62468 comments](https://github.com/airbytehq/airbyte/issues/62468)]
e
note that I'm using helm @kapa.ai
k
A message from kapa.ai
For Helm-based OSS installs, the only concrete behavior in the sources is: - The error appears whenever
airbyte-auth-secrets
is missing the keys
dataplane-client-id
/
dataplane-client-secret
. [[1.6.0 chart issue](https://github.com/airbytehq/airbyte/issues/62468); [1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - When users pre‑create and manage
airbyte-auth-secrets
themselves (typically just for
instance-admin-email
/
instance-admin-password
), the bootloader and workload pods cannot reliably populate or use the dataplane keys, causing your exact error. [[1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - A working pattern reported in those issues is to let the chart/bootloader manage
airbyte-auth-secrets
entirely
, instead of managing it yourself. Deleting the manually created secret and the related Helm values allows Airbyte to recreate the secret and add
dataplane-client-id
and
dataplane-client-secret
, after which the workload-launcher runs successfully. [[1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] The knowledge sources do not document: - Any explicit values you should set for
dataplane-client-id
/
dataplane-client-secret
in an OSS Helm deployment. - A supported way to hand‑craft those values for OSS. In fact, there are reports that even when users manually set them in the secret, the bootloader overwrites them. [[bootloader overwrite](https://github.com/airbytehq/airbyte/issues/64159); [62468 comments](https://github.com/airbytehq/airbyte/issues/62468)] Because of that, I can’t reliably tell you “set them to X” from the docs: they appear to be generated and managed internally, not user‑provided values in OSS. Given you are on Helm: 1. If you currently define a custom auth secret in
values.yaml
, e.g.:
Copy code
yaml    auth:      enabled: true      instanceAdmin:        secretName: airbyte-auth-secrets        emailSecretKey: instance-admin-email        passwordSecretKey: instance-admin-password
this configuration is exactly the one that has triggered the error for others. [[1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] 2. The only workaround clearly shown to work in the sources is: - Delete the existing
airbyte-auth-secrets
from your namespace. - Remove or comment out the
auth.instanceAdmin.secretName
/
emailSecretKey
/
passwordSecretKey
overrides from your Helm values so the chart can manage that secret itself. [[1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - Re‑deploy the chart so the bootloader recreates
airbyte-auth-secrets
and injects
dataplane-client-id
/
dataplane-client-secret
. The sources do not describe a safe, documented way (for OSS + Helm) to keep your own
airbyte-auth-secrets
and manually supply dataplane values, so I can’t recommend a specific
values.yaml
pattern for that scenario based on the available information.
e
I do manage things externally using aws secret manager. What will happen to instance-admin-email and other things I already have setup? @kapa.ai
k
A message from kapa.ai
The behavior you’re asking about (what happens to
instance-admin-email
etc. when you manage secrets externally) is not clearly documented in the official docs, and the only concrete evidence we have comes from GitHub issues. From those: - When users pre‑create and externally manage the
airbyte-auth-secrets
Secret (for example, via AWS Secrets Manager + ExternalSecrets), Airbyte’s bootloader still tries to manage data‑plane–related keys (
dataplane-client-id
,
dataplane-client-secret
) inside that same secret. [[1.6.0 dataplane issue](https://github.com/airbytehq/airbyte/issues/62468); [auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159)] - At least in some versions, the bootloader overwrites credentials in the auth secret when it initializes the dataplane. This is why a user explicitly asked: “is it possible to define dataplane-client credentials without rewriting the auth secret?” [[1.6.0 dataplane issue](https://github.com/airbytehq/airbyte/issues/62468); [auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159)] - Another user reported that when they had
airbyte-auth-secrets
pre‑created and managed externally, the secret was not being updated with the dataplane keys, even though the role was configured to allow it. As a workaround they: - Commented out the
global.auth.instanceAdmin.secretName/emailSecretKey/passwordSecretKey
overrides in Helm (which pointed to their managed secret), and - Deleted the existing
airbyte-auth-secrets
. After that, Airbyte re‑created
airbyte-auth-secrets
itself, “somehow it kept the same values for the instance admin keys, but also created the dataplane-client-id and dataplane-client-secret,” and the system became healthy. [[abctl 1.6.0 issue](https://github.com/airbytehq/airbyte/issues/58587)] However, there is no explicit documentation of: - A supported pattern for using AWS Secrets Manager / ExternalSecrets to manage
instance-admin-email
and
instance-admin-password
while also letting Airbyte manage dataplane keys in the same secret. - Any guarantee that the bootloader will always preserve externally provided
instance-admin-*
keys when it (re)creates or updates
airbyte-auth-secrets
. So, based on what we do know: 1. If you keep managing
airbyte-auth-secrets
entirely via AWS Secrets Manager: - The knowledge sources do not state that your existing
instance-admin-email
and
instance-admin-password
will be preserved. - There is evidence that bootloader may overwrite or fail to update keys in that secret depending on version and initialization flow. [[auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159); [abctl 1.6.0 issue](https://github.com/airbytehq/airbyte/issues/58587)] 2. If you let Airbyte/bootloader create and manage `airbyte-auth-secrets`: - A user observed that their instance admin keys “somehow” stayed the same when the secret was re‑created, but that’s an observation, not a documented contract. [[abctl 1.6.0 issue](https://github.com/airbytehq/airbyte/issues/58587)] - This is currently the only configuration in the sources that clearly results in valid dataplane keys plus working workload‑launcher. Because the docs and issues do not describe a stable, supported interaction between ExternalSecrets/AWS Secrets Manager and the
airbyte-auth-secrets
bootloader logic, I can’t confidently say that your externally managed
instance-admin-email
and related fields will be “safe” or untouched. The safest interpretation from the available information is: -
airbyte-auth-secrets
is expected to be platform‑managed rather than fully controlled by an external secret manager. - Using AWS Secrets Manager to manage other Airbyte secrets (for connectors etc.) is documented and supported, but `airbyte-auth-secrets`’s internal structure and lifecycle are not documented as user‑configurable. [[Secret management values](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] If you must keep using AWS Secrets Manager for admin credentials, the current knowledge sources don’t provide a clear, supported way to do that without risking conflicts with bootloader’s management of the same secret.
e
is airbyte creating the dataplane secrets somewhere? how to get it and add to my secrets manager? @kapa.ai
k
A message from kapa.ai
Airbyte does create dataplane credentials, but how and where is only partially visible in the sources, and there is no documented, supported way for OSS Helm users to “export” them into an external secrets manager. What we know from the docs and issues: 1. Bootloader generates / manages dataplane credentials - The bootloader has a
DataplaneInitializer
that creates and/or updates dataplane client credentials and writes them into Kubernetes secrets. [[bootloader DataplaneInitializer](https://github.com/airbytehq/airbyte/issues/62468); [bootloader overwrite](https://github.com/airbytehq/airbyte/issues/64159)] - For Enterprise Flex data planes, the control plane issues
DATA_PLANE_CLIENT_ID
and
DATA_PLANE_CLIENT_SECRET
, and you store them in a K8s secret like
airbyte-config-secrets
(or equivalent), then reference them from
values.yaml
. [[Enterprise data plane](https://docs.airbyte.com/platform/next/enterprise-flex/data-plane#step-3)] - In OSS Helm v1, dataplane-related values are embedded into the auth / workload config, and several issues show that
DATAPLANE_CLIENT_ID
/
DATAPLANE_CLIENT_SECRET
(or
dataplane-client-id
keys) are injected via
airbyte-auth-secrets
. [[1.6.0 dataplane issue](https://github.com/airbytehq/airbyte/issues/62468); [config map keys](https://github.com/airbytehq/airbyte/issues/66546)] 2. Location in OSS Helm installs - Errors like
couldn't find key dataplane-client-id in Secret <ns>/airbyte-auth-secrets
show that, in your OSS + Helm setup, the bootloader and workload-launcher expect these values inside the
airbyte-auth-secrets
K8s Secret. [[1.6.x dataplane error](https://github.com/airbytehq/airbyte/issues/58587); [1.6.0 chart issue](https://github.com/airbytehq/airbyte/issues/62468)] - When users do not pre-create that secret and let Airbyte/bootloader manage it, it ends up containing
dataplane-client-id
and
dataplane-client-secret
, and the system works. [[1.6.x dataplane error](https://github.com/airbytehq/airbyte/issues/58587)] 3. Interaction with external secret managers is not documented - The docs describe using external secret managers (AWS Secrets Manager, etc.) for general secrets, but they do not document a pattern for syncing the auto-generated dataplane credentials into an external manager. [[Secret management values](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] - Users who pre‑create/manage
airbyte-auth-secrets
externally (including dataplane keys) have run into problems: the bootloader may overwrite or fail to populate those keys. [[auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159); [1.6.x dataplane error](https://github.com/airbytehq/airbyte/issues/58587); [62468 comments](https://github.com/airbytehq/airbyte/issues/62468)] - There is no doc or issue comment describing an official “read the generated dataplane credentials and push them to AWS Secrets Manager” pattern. Given that, the best we can say from the sources: - Yes, Airbyte (via bootloader) creates/manage dataplane credentials inside Kubernetes secrets, and in OSS Helm v1 it uses
airbyte-auth-secrets
as the place those keys are expected. [[1.6.0 chart issue](https://github.com/airbytehq/airbyte/issues/62468); [auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159)] - The knowledge sources do not document a supported method to: - programmatically retrieve those credentials from Airbyte in a stable way (apart from inspecting the K8s Secret directly), and then - sync them into an external secrets manager such as AWS Secrets Manager. So, within the limits of the docs/issues: - You can manually read whatever the bootloader has stored in the
airbyte-auth-secrets
(or the relevant dataplane secret in Enterprise Flex) and then put those values into AWS Secrets Manager yourself, but that is not described or guaranteed by the official docs. - There is no documented API or Helm option that says “export dataplane credentials to an external secrets manager” for OSS core. If you need an officially supported pattern for externalizing dataplane credentials, the existing documentation and issues do not provide one; you’d need guidance from the Airbyte team beyond what’s available in these sources.
e
So I need to remove all my airbyte-auth-secrets config? Will my current secrets stay valid? @kapa.ai
k
A message from kapa.ai
Based on the information in the issues and docs, there are two separate questions here: 1. “Do I need to remove all my
airbyte-auth-secrets
config?”
2. “Will my current secrets stay valid?” ### 1. Do you need to remove your
airbyte-auth-secrets
config? All the concrete troubleshooting we have for the
dataplane-client-id
error with Helm v1 shows it happening when
airbyte-auth-secrets
is pre‑created and managed externally
(via values or an external secret manager). [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587); [dataplane missing key](https://github.com/airbytehq/airbyte/issues/62468)] In that 1.6.x case, the user resolved the exact same error by: - Commenting out the Helm values that override the auth secret, e.g.:
Copy code
yaml   global:     auth:       instanceAdmin:         secretName: airbyte-auth-secrets         emailSecretKey: instance-admin-email         passwordSecretKey: instance-admin-password
- Deleting the existing
airbyte-auth-secrets
Secret, and - Letting Airbyte/bootloader recreate and populate it. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] After that,
airbyte-auth-secrets
contained the required
dataplane-client-id
/
dataplane-client-secret
, and workload‑launcher became healthy. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] So, based on the sources, the only clearly working pattern when you hit this error is: - Stop managing
airbyte-auth-secrets
via Helm values / external manager, and - Allow Airbyte to manage that secret itself. The docs and issues do not show a supported way to keep
airbyte-auth-secrets
fully external and satisfy the dataplane keys for OSS Helm v1. ### 2. Will your existing admin secrets stay valid? This is where the sources are incomplete: - In the 1.6.x workaround above, the user reports that after deleting their managed secret and redeploying, Airbyte created a new
airbyte-auth-secrets
and “**somehow it kept the same values for the instance admin keys**, but also created the dataplane-client-id and dataplane-client-secret.” [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] That’s observed behavior in that version, not a documented guarantee. - Another issue shows that the bootloader may overwrite values in the auth secret when doing dataplane initialization, which is why users asked for it not to rewrite credentials. [[auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159); [62468 comments](https://github.com/airbytehq/airbyte/issues/62468)] - The docs about authentication and
airbyte-auth-secrets
only describe how to: - Read the generated admin credentials, or - Override the admin password (and optionally client id/secret) by creating the secret yourself. [[Auth docs](https://docs.airbyte.com/platform/deploying-airbyte/integrations/authentication)] They do not state what happens if the platform later recreates or modifies that secret while also managing dataplane keys. Because of this, the knowledge sources: - Do not clearly guarantee that your current
instance-admin-email
/
instance-admin-password
(or other keys) will remain unchanged if you delete your managed secret and let Airbyte recreate it. - Only show one user’s report where they happened to remain the same after such a reset. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] So, from the available information: - To fix the dataplane error with Helm v1, the pattern supported by the issues is to remove your
airbyte-auth-secrets
overrides and delete the secret
, letting Airbyte recreate it. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - Whether your current admin credentials “stay valid” after that is not guaranteed by the docs; in at least one case they did, but another issue shows bootloader can overwrite auth secret content. [[1.6.0 abctl issue](https://github.com/airbytehq/airbyte/issues/58587); [auth secret overwrite](https://github.com/airbytehq/airbyte/issues/64159)] Given this, if you proceed with that workaround, you should plan as if your admin credentials might change and be ready to: - Read the new values from
airbyte-auth-secrets
after redeploy (via
kubectl get secret ...
as in the auth docs), and - Update any external references (e.g., your own Secret Manager) accordingly. [[Auth docs](https://docs.airbyte.com/platform/deploying-airbyte/integrations/authentication)]
e
Does using helm v2 solve this issue? @kapa.ai
k
A message from kapa.ai
It likely improves the situation, but the knowledge sources don’t state clearly that “Helm chart V2 solves this issue” in all OSS cases. What we do know: - The
dataplane-client-id
/
dataplane-client-secret
problems are explicitly reported for Helm chart V1 and abctl installs around 1.6.x and 1.8.x, especially when
airbyte-auth-secrets
is pre‑created / externally managed. [[1.6.0 chart issue](https://github.com/airbytehq/airbyte/issues/62468); [1.6.x abctl issue](https://github.com/airbytehq/airbyte/issues/58587)] - The behavior of the bootloader around dataplane credentials was changed so that if valid credentials already exist in the secret, it will skip dataplane initialization (and thus stop constantly overwriting). This fix is available starting in release 1.8.0. [[bootloader fix](https://github.com/airbytehq/airbyte/issues/64159)] - Helm chart V2 is the new chart line recommended for 1.6.0+ Helm installs and is generally more consistent with best practices, but the V2 docs and implementation guides do not explicitly call out this dataplane/auth‑secret issue as “fixed by V2”. They mainly focus on changes in values structure, external DB/storage, and secret manager configuration. [[Chart V2 enterprise upgrade](https://docs.airbyte.com/platform/enterprise-setup/chart-v2-enterprise); [Next chart V2](https://docs.airbyte.com/platform/next/enterprise-setup/chart-v2-enterprise)] From this, the safest statement based on the sources is: - Moving to a newer Airbyte version (≥1.8.0) that contains the bootloader fix should reduce problems with the auth secret constantly being rewritten for dataplane credentials, regardless of chart version. [[bootloader fix](https://github.com/airbytehq/airbyte/issues/64159)] - Using Helm chart V2 is recommended for new/updated Helm deployments and may avoid some of the chart‑level rough edges seen in V1, but the docs and issues provided do not explicitly guarantee that simply switching to V2 will, by itself, eliminate all
dataplane-client-id
missing/secret‑management conflicts. So: upgrading to a version that includes the bootloader change (1.8.0+) and using chart V2 is likely to behave better than older V1 setups, but the knowledge sources do not explicitly state “Helm V2 fixes this issue,” so I cannot confirm that as a documented fact.