https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> question in thread
# replication-ask-ai
t
@kapa.ai question in thread
--- Title: Airbyte 2.0.1 OSS: Workload init container fails with SecretCoordinateException when using TESTING_CONFIG_DB_TABLE secret persistence Description: After upgrading to Airbyte 2.0.1 (Helm chart 2.0.19), programmatic source creation via API works, but connection creation fails because the discover workload's init container can't hydrate secrets. Environment: - Airbyte version: 2.0.1 (Helm chart 2.0.19) - Deployment: Kubernetes (k3d local dev) - Secret persistence: TESTING_CONFIG_DB_TABLE - Database: External PostgreSQL (Supabase) What works: - Creating sources via /api/v1/sources/create — secrets are written to the secrets table in db-airbyte - Creating sources/connections via the Airbyte UI — works fine - Staging environment using google_secret_manager — works fine What fails: - Creating connections via API → triggers discover workload → init container crashes with: secrets.persistence.SecretCoordinateException: That secret was not found in the store! Coordinate: airbyte_workspace_<workspace-id>_secret_<secret-id>_v1 Investigation: - The secret coordinate exists in the secrets table (verified via SQL query) - The workload init container has SECRET_PERSISTENCE: TESTING_CONFIG_DB_TABLE and correct DATABASE_URL - The RealSecretsHydrator in the init container can't find the secret despite it being present in the DB - Setting secretsManager.enabled: false or SECRET_PERSISTENCE: '' in Helm values doesn't change the behavior — workload pods still get TESTING_CONFIG_DB_TABLE - Even after dropping and recreating the db-airbyte database and reinstalling Helm, the issue persists Expected behavior: The init container should be able to read secrets from the secrets table when SECRET_PERSISTENCE=TESTING_CONFIG_DB_TABLE. Workaround: Creating sources and connections through the Airbyte UI instead of the API. Questions: 1. Is TESTING_CONFIG_DB_TABLE still supported in Airbyte 2.0 for programmatic source/connection creation? 2. Is there a recommended secret persistence mode for local/OSS development that works with the workload architecture? 3. Is the RealSecretsHydrator looking in a different table (e.g., secret_storage) instead of the legacy secrets table?
k
Based on the knowledge sources available, I can provide some partial answers to your questions, though the sources don't fully address all the specifics of your situation. What the Knowledge Sources Tell Us On 
TESTING_CONFIG_DB_TABLE
support: The knowledge sources confirm that 
TESTING_CONFIG_DB_TABLE
is a recognized secret persistence type in Airbyte 2.0, listed as an option for 
SECRET_PERSISTENCE
— described as "use the database as a test." [Configuring Airbyte] However, the sources do not explicitly confirm whether it is fully supported for programmatic source/connection creation in the workload architecture of v2.0. On how secrets work in Airbyte: Airbyte stores secrets using a coordinate system of the form 
airbyte_workspace_<workspace-id>_secret_<uuid>_v1
. Upon any operation, the init container reads the obfuscated JSON spec, extracts the secret coordinate, and hydrates the correct secret value before passing it to consumers. [Secrets] On a related init container issue with secret persistence env vars: There is a known bug (GitHub issue #69233) where the 
checkEnvVars
and 
discoverEnvVars
beans in 
airbyte-workload-launcher
only inject 
airbyteMetadataEnvMap
into check/discover pods, missing 
secretPersistenceEnvMap
and 
secretPersistenceSecretsEnvMap
. This means discover/check pods may not receive the correct secret persistence configuration environment variables, even though sync pods work correctly. [Init container Azure clientSecret issue] This is very likely related to your issue — if the discover workload's init container isn't receiving the correct 
DATABASE_URL
or 
SECRET_PERSISTENCE
env vars (despite them appearing set), it could be looking at the wrong database or failing to connect. Recommendations 1. Inspect the actual env vars on the failing init container pod to verify that 
DATABASE_URL
and 
SECRET_PERSISTENCE
are correctly set to what you expect — the bug above suggests they may not be propagated correctly to discover pods. 2. For local/OSS development, the knowledge sources do not explicitly recommend a specific secret persistence mode. However, given the known issues with 
TESTING_CONFIG_DB_TABLE
in the workload architecture, using an external secret manager (AWS, GCP, Azure Key Vault, or HashiCorp Vault) is what Airbyte highly recommends for production-like setups. [Secret Management] 3. On the 
secret_storage
table question: The knowledge sources do not contain enough information to confirm whether 
RealSecretsHydrator
changed its target table in v2.0. I'd recommend opening or searching for a GitHub issue specifically about 
TESTING_CONFIG_DB_TABLE
+ workload init containers in Helm chart v2, as the root cause may be the same env var propagation bug documented in issue #69233. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
It looks like your message came through as "question in thread" without the actual question content. Could you please share the specific question you'd like help with? I'm happy to assist with anything related to your Airbyte setup or the issue described in the conversation. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
2 Views
Open in Slack
PreviousNext
Powered by