<@U04S8PLJWMT> I get this error when doing check c...
# replication-ask-ai
r
@kapa.ai I get this error when doing check connection api call: {'jobInfo': {'id': 'c7b8b3f5-e844-453e-ad75-acbd03bef020', 'configType': 'check_connection_source', 'configId': 'Optional[e7778cfc-e97c-4458-9ecb-b4f2bba8946c]', 'createdAt': 1776751637510, 'endedAt': 1776751650725, 'succeeded': False, 'connectorConfigurationUpdated': False, 'logType': 'formatted', 'logs': {'events': [], 'logLines': []}, 'failureReason': {'failureOrigin': 'airbyte_platform', 'externalMessage': 'Workload failed, source: airbyte_platform', 'internalMessage': 'Init container error encountered while hydrating secrets for id: e7778cfc-e97c-4458-9ecb-b4f2bba8946c_4c78408f-89d7-4789-9d77-81b6bb3e8568_0_check. Encountered exception of type: class secrets.persistence.SecretCoordinateException. Exception message: That secret was not found in the store! Coordinate: airbyte_workspace_5479d125-ccb5-4efe-868d-670062587f92_secret_c031de9e-c133-414a-a3c6-538166ff8688_v3.', 'timestamp': 1776751650605}}}
k
The error you're seeing indicates that during a check connection operation, the init container tried to hydrate (retrieve) a secret from your configured secret store, but the secret coordinate was not found:
Copy code
That secret was not found in the store! 
Coordinate: airbyte_workspace_5479d125-ccb5-4efe-868d-670062587f92_secret_c031de9e-c133-414a-a3c6-538166ff8688_v3
What's happening Airbyte stores connector credentials by obfuscating them — it writes the actual secret to your configured secret persistence layer (e.g., AWS Secrets Manager, GCP Secret Manager, etc.) and stores only a coordinate reference in its database. [Secrets docs] When a check/sync runs, the init container fetches the secret using that coordinate. The error means the coordinate
_v3
exists in Airbyte's database, but the actual secret value is missing from your secret store. Likely causes 1. Secret was manually deleted from your external secret manager (AWS/GCP/Azure/Vault) without updating Airbyte's database. 2. Secret manager connectivity issue — the init container cannot authenticate or connect to the secret store. This is a known issue in Helm chart v2.x where the init container (used for check/discover operations) does not receive the secret manager credentials (e.g.,
AWS_SECRET_MANAGER_ACCESS_KEY_ID
), even if sync pods do. [AWS secret manager issue; Azure init container issue] 3. Version mismatch — the secret was created at
_v3
but a different version exists in the store. What to check • Verify the secret exists in your cloud provider's console (e.g., AWS Secrets Manager in your configured region) with the coordinate name shown in the error. [Verification steps] • Check init container credentials: Inspect the init container's environment variables to confirm it has the necessary secret manager credentials. If they're missing, this is the known bug in
airbyte-workload-launcher
where
checkEnvVars
and
discoverEnvVars
don't include
secretPersistenceEnvMap
or
secretPersistenceSecretsEnvMap
. [Azure init container issue] • Re-save your connector configuration in the Airbyte UI — this will write a new secret version to the store and update the coordinate reference. If you're on Helm chart v2.x with an external secret manager, this is a known open bug. The workaround is to ensure the init container has the required credentials, or to use `instanceProfile`/workload identity authentication if your cloud environment supports it. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
I am not using external secret manager
@kapa.ai I am not using external secret manager
k
Since you're not using an external secret manager, Airbyte's default behavior is to store connector secrets on your configured database (in plain-text, unencrypted) or as Kubernetes secrets on your cluster. [Secret Management] In that case, the error —
That secret was not found in the store!
— suggests that the secret coordinate stored in Airbyte's database is pointing to a secret that no longer exists in the default store. This can happen if: 1. The database was partially reset or migrated, causing the secret data to be lost while the coordinate reference remained. 2. The connector configuration is in an inconsistent state — the coordinate reference exists but the underlying secret entry was deleted or corrupted. Recommended fix Re-save your connector configuration: Go to the Airbyte UI, open the source connector that's failing, re-enter the credentials, and save. This will write a fresh secret to the store and update the coordinate reference to a new version (e.g.,
_v4
). If the issue persists after re-saving, it may indicate a deeper database consistency problem, and I'd recommend reaching out to Airbyte support with your deployment details. > Note: The knowledge sources do not explicitly cover troubleshooting steps for the default (non-external) secret store beyond what's described above, so I cannot provide more specific guidance on diagnosing the root cause in that configuration. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai getting error when trying to install:
Copy code
ERROR   unable to install airbyte chart: unable to install helm: pre-upgrade hooks failed: 1 error occurred:
          	* pod airbyte-abctl-bootloader failed
k
This is a known and actively investigated issue with
abctl
installations. Here's what's known about it: What's happening The
airbyte-bootloader
is the first service to start during installation. When it fails, the Helm pre-upgrade hook fails and the entire installation is blocked. The failure is generally related to the bootloader not being able to connect to the
airbyte-db
service, though there can be other causes. [Troubleshooting abctl] Known root causes 1. Duplicate key in the remote connector registry — A known instance of this error was caused by
source-sap-hana
and
source-sap-hana-enterprise
sharing the same
sourceDefinitionId
in the remote registry. The bootloader fetches the live remote registry at startup regardless of chart version, so pinning to an older chart version doesn't help in this case. [Bootloader failed issue] 2. JSON parse error in the database — Another known cause is a
JsonParseException: Invalid numeric value: Leading zeroes not allowed
error in the bootloader logs, which can be fixed by cleaning up the dataplane tables: [Migration failing issue]kind export kubeconfig -n airbyte-abctl kubectl exec -it -n airbyte-abctl airbyte-db-0 -- /bin/bash psql -U airbyte db-airbyte delete from dataplane_client_credentials; delete from dataplane; \q exitThen re-run
abctl local install
. 3. M4 Mac-specific issue — Users on Apple M4 chips have also reported this error. This is under investigation. [M4 macOS issue] 4. Disk/image loading issues — An image import failure into the Kind node can cause downstream pod failures. Check disk space with
df -h
and Docker usage with
docker system df
. [Bootloader failed issue] General troubleshooting steps 1. Run with verbose logging to see the full bootloader logs:abctl local install --verbose 2. Check bootloader pod logs directly:kubectl logs airbyte-abctl-bootloader -n airbyte-abctl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig 3. Try a clean reinstall:abctl local uninstall abctl local install -v 4. Try pinning to a known working chart version (some users reported success with 1.7.2):abctl local install --chart-version=1.7.2 Could you share the full verbose output (
abctl local install --verbose
)? The bootloader logs will reveal the specific root cause in your case. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.