Hello guys i am trying to setup airbyte using helm...
# replication-troubleshooting
j
Hello guys i am trying to setup airbyte using helm v2 in my helmrelease file, I want to use vault as secretmanager. I am following what is given and tried many more things then too when am creating any source in airbyte getting an error:- Configuration check failed Workload failed, source: airbyte_platform Internal message: Init container error encountered while hydrating secrets for id: 6e2-b3-4c7-xxx-4499ad_0_check. Encountered exception of type: class secrets.persistence.SecretCoordinateException. Exception message: That secret was not found in the store! Coordinate: airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_f1fc-6e3c-4877-xxx-933_v1. Failure origin: airbyte_platform can someone help me this is really urgent thanks in advance
k
This error means Airbyte is trying to read a secret from the configured secret manager (Vault in your case), but that secret coordinate does not exist there: >
That secret was not found in the store! Coordinate: airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_..._v1
A few key points from the docs that are relevant: 1. How Airbyte uses secret coordinates - When a connector has a field marked
airbyte_secret: true
, Airbyte: - Writes the real value to the configured secret store under a generated coordinate like
airbyte_workspace_<workspace-id>_secret_<uuid>_<version>
. - Stores only that coordinate in the DB / config JSON. [[Secrets usage](https://docs.airbyte.com/platform/understanding-airbyte/secrets)] - For operations like Check or Discover that are not tied to a workspace, the workspace id in the coordinate is
00000000-0000-0000-0000-000000000000
. [[Secrets usage](https://docs.airbyte.com/platform/understanding-airbyte/secrets)] Your failing coordinate is exactly one of those “non‑workspace” secrets. 2. Configuring Vault as the connector secret manager (Helm v2) For Helm v2, the
values.yaml
must have the
global.secretsManager
block set to
VAULT
, for example: [[Secret values 1.8](https://docs.airbyte.com/platform/1.8/deploying-airbyte/integrations/secrets#values); [Secret values next](https://docs.airbyte.com/platform/next/deploying-airbyte/integrations/secrets#values)]
Copy code
yaml    global:      secretsManager:        type: VAULT        secretsManagerSecretName: "airbyte-config-secrets" # Name of your Kubernetes secret        vault:          address: "https://<your-vault-url>"   # e.g. <https://my-vault.vault.azure.net/> or your Vault endpoint          prefix: "kv/airbyte/"                 # or whatever path you use, must end with /          authToken: ""                         # OR          authTokenSecretKey: ""                # key in the Kubernetes secret that holds the token
And you must have a Kubernetes secret named
airbyte-config-secrets
(or whatever you set in
secretsManagerSecretName
) that actually contains the Vault token if you use
authTokenSecretKey
. [[Secret values deploy](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] 3. Common causes of “secret not found in the store” with Vault Based on the docs, the following misconfigurations will lead to exactly this symptom: -
global.secretsManager.type
is not set to
VAULT
(or is still the default), so Airbyte is writing coordinates but not actually able to write/read from Vault. -
vault.prefix
is wrong (e.g. missing the engine name or trailing slash), so Airbyte writes to one path and then looks up in another. -
vault.address
or the Vault token (via
authToken
/
authTokenSecretKey
) is invalid, so writes fail and the coordinate is created in the DB but no secret is stored. - The
secretsManagerSecretName
Kubernetes secret is missing or does not contain the key referenced by
authTokenSecretKey
. 4. Important warning when changing secret manager The docs explicitly state: changing the connector secret storage configuration **will cause all existing connectors to fail**; you must recreate them. [[Secret values deploy](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] So if you switched to Vault after creating some sources/destinations, you need to: - Delete those connectors. - Recreate them after Vault is correctly configured. --- ### What to check in your setup Given the above, here’s what you should verify in your HelmRelease (values) and cluster: 1. In your HelmRelease values, under `global.secretsManager`: -
type: VAULT
-
secretsManagerSecretName: "airbyte-config-secrets"
(or your chosen name) -
vault.address
,
vault.prefix
, and either
vault.authToken
or
vault.authTokenSecretKey
are set. 2. In Kubernetes: - The secret named in
secretsManagerSecretName
exists. - If you use
authTokenSecretKey
, that key exists in the secret and contains a valid Vault token with read/write permissions on the
prefix
path. 3. After fixing the configuration: - Restart / upgrade the Airbyte release so all pods pick up the new config. - Delete and recreate the failing source so its secrets are freshly written to Vault. If you can share the
global.secretsManager
section from your HelmRelease (with sensitive values redacted), I can help you pinpoint what in that block doesn’t match the documented patterns.
j
@kapa.ai Error: couldn't find key VAULT_TOKEN in Secret airbyte/airbyte-airbyte-secrets
k
A message from kapa.ai
The error means Kubernetes is looking for a key named
VAULT_TOKEN
inside the secret
airbyte-airbyte-secrets
in the
airbyte
namespace, and that key does not exist. From the Helm v2 docs, when using Vault as the secrets manager you must: 1. Create a Kubernetes secret that contains your Vault token, e.g.
airbyte-config-secrets
, with a key that matches what you configure in `values.yaml`:
Copy code
yaml    apiVersion: v1    kind: Secret    metadata:      name: airbyte-config-secrets    type: Opaque    stringData:      VAULT_TOKEN: "<your-vault-token>"
Apply it with:
Copy code
bash    kubectl apply -f secrets.yaml -n airbyte
or create it directly with
kubectl create secret ...
. [[Preconfiguring secrets](https://docs.airbyte.com/platform/deploying-airbyte/creating-secrets)] 2. Reference that secret and key correctly in your Helm values for Vault. For Helm v2, the pattern is: [[Vault values](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)]
Copy code
yaml    global:      secretsManager:        type: VAULT        secretsManagerSecretName: "airbyte-config-secrets"  # name of the K8s secret        vault:          address: "https://<your-vault-url>"          prefix: "kv/airbyte/"          authTokenSecretKey: "VAULT_TOKEN"                  # key inside the K8s secret
Given your error: - Either the secret name is
airbyte-airbyte-secrets
instead of
airbyte-config-secrets
, or - The key inside that secret is not
VAULT_TOKEN
. To fix: 1. Check the secret contents:
Copy code
bash    kubectl get secret airbyte-airbyte-secrets -n airbyte -o yaml
2. Ensure it has a
VAULT_TOKEN
key (or change your
authTokenSecretKey
in
values.yaml
to match the existing key). 3. Ensure
secretsManagerSecretName
in your values matches the actual secret name. 4. Re‑deploy the HelmRelease so Airbyte picks up the corrected secret.
j
@kapa.ai if my vault cli path:- -mount="kv/non-prod/cdp" "gke-use1-symphony-dev/airbyte" what will be prefix to use above
k
A message from kapa.ai
The docs only state that
vault.prefix
should be “the vault path prefix” and give examples like: > Should follow the format
<engine>/<directory>/
, for example
kv/airbyte/
or
secret/airbyte/
. [[Configuring Airbyte](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte)] They do not describe how to derive this from a specific
vault kv
CLI command like the one you shared, so I can’t reliably compute the exact prefix from your CLI path. What you can infer from the docs: -
vault.prefix
must be the static part of the path under which Airbyte will create keys, i.e.
<engine>/<directory>/
. [[Configuring Airbyte](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte)] - Airbyte will then append its own secret coordinates (like
airbyte_workspace_..._v1
) under that prefix. So, for your case, you should choose a prefix that matches the engine and directory you actually want Airbyte to use, in the same pattern as the examples, e.g.:
Copy code
yaml global:   secretsManager:     type: VAULT     vault:       address: "https://<your-vault-url>"       prefix: "<your-engine>/<your-directory>/"   # e.g. kv/airbyte/
Because the docs don’t show how
-mount="kv/non-prod/cdp" "gke-use1-symphony-dev/airbyte"
maps into
<engine>/<directory>/
, I can’t say with confidence whether your prefix should be
kv/non-prod/cdp/
,
kv/non-prod/cdp/gke-use1-symphony-dev/
, or something else. To proceed safely: 1. Identify in your Vault setup which part is the engine name (e.g.
kv
or
kv2
) and which part is the directory you want Airbyte to own (e.g.
airbyte/
). 2. Set
vault.prefix
to exactly
<engine>/<directory>/
as per the examples. [[Configuring Airbyte](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte)] 3. After setting it, create a connector and then inspect Vault to see where the new secret appears; adjust the prefix if needed. If you can share (in redacted form) how your KV engine is enabled in Vault (e.g.
vault secrets enable -path=... kv
), that would let you map it to the
<engine>/<directory>/
pattern, but that mapping is not documented in the Airbyte docs themselves.
j
@kapa.ai other than secretmanager in values.yaml is there any other thing needed to for connecting Vault and airbyte
k
A message from kapa.ai
Based on the docs, everything needed for Vault integration is configured through: 1.
values.yaml
(Helm)
– via `global.secretsManager`:
Copy code
yaml    global:      secretsManager:        type: VAULT        secretsManagerSecretName: "airbyte-config-secrets"  # K8s secret name        vault:          address: ""          # Vault URL          prefix: ""           # e.g. kv/airbyte/          authToken: ""        # OR          authTokenSecretKey: ""  # key in the K8s secret
[[Secret values deploy](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] 2. A Kubernetes secret named in
secretsManagerSecretName
that actually holds the Vault token if you use
authTokenSecretKey
. [[Secret values deploy](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] There are no additional Airbyte‑specific settings for Vault mentioned beyond: - Setting
global.secretsManager.type
to
VAULT
. - Providing
vault.address
,
vault.prefix
, and token (
authToken
or
authTokenSecretKey
). - Ensuring the referenced Kubernetes secret exists and contains the token. The operator‑guide also lists the corresponding environment variables (
VAULT_ADDRESS
,
VAULT_PREFIX
,
VAULT_AUTH_TOKEN
, etc.), but when using Helm v2 you are expected to configure these via
values.yaml
, not separately. [[Configuring Airbyte](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte)] If you’ve set
global.secretsManager
correctly and created the Kubernetes secret with the Vault token, the docs do not indicate any other required configuration for connecting Airbyte to Vault.
j
@kapa.ai i am using version: "2.0.18
k
A message from kapa.ai
For Airbyte 2.0.18 with Helm chart v2, the Vault setup is still driven entirely through the
global.secretsManager
block in `values.yaml`; there isn’t any extra Vault‑specific toggle outside of that in the docs. For chart v2, the documented pattern is: [[Chart v2 secrets](https://docs.airbyte.com/platform/deploying-airbyte/chart-v2-community#update-your-valuesyaml-file); [Vault values](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)]
Copy code
yaml global:   secretsManager:     enabled: false              # set to true if you want to force-enable; some examples omit it     type: VAULT                 # one of: VAULT, GOOGLE_SECRET_MANAGER, AWS_SECRET_MANAGER, AZURE_KEY_VAULT, TESTING_CONFIG_DB_TABLE     secretName: "airbyte-config-secrets"   # (or secretsManagerSecretName in some snippets)      vault:       address: "https://<your-vault-url>"       prefix: "kv/airbyte/"     # your engine/path prefix       authToken: ""             # OR       authTokenSecretKey: ""    # key inside the K8s secret named above
Key things you must have for 2.0.18: 1. Correct
global.secretsManager
block
as above, with
type: VAULT
and
vault.address/prefix/authToken*
set. [[Chart v2 secrets](https://docs.airbyte.com/platform/deploying-airbyte/chart-v2-community#update-your-valuesyaml-file)] 2. Kubernetes secret for the Vault token The secret name must match `secretName`/`secretsManagerSecretName`, and it must contain the key you reference in
authTokenSecretKey
(for example
VAULT_TOKEN
). [[Vault values](https://docs.airbyte.com/platform/deploying-airbyte/integrations/secrets#values)] 3. No additional Airbyte flags are required The operator‑guide lists the underlying env vars (
VAULT_ADDRESS
,
VAULT_PREFIX
,
VAULT_AUTH_TOKEN
, etc.), but with Helm v2 you configure them only via `values.yaml`; there is no extra Vault switch beyond
global.secretsManager
. [[Configuring Airbyte](https://docs.airbyte.com/platform/operator-guides/configuring-airbyte)] If you share the exact
global.secretsManager
section from your 2.0.18 values (with secrets redacted), I can help check it against the documented v2 schema.