https://linen.dev logo
Join Slack
Powered by
Hey <@U04SE163WC8> how do i setup google secrets m...
# ask-ai
i
Hey @kapa.ai how do i setup google secrets manager for airbyte on kubernetes ?
u
To set up Google Secrets Manager for Airbyte on Kubernetes, you need to follow these steps: 1. Setting Up or Updating Kubernetes Secrets For Google Secret Manager, you may use existing credentials or create new ones. Apply a Kubernetes manifest like below, replacing 
<CREDENTIALS_JSON_BLOB>
with your GCP credentials JSON blob:
Copy code
yaml    apiVersion: v1      kind: Secret      metadata:        name: gcp-cred-secrets      type: Opaque      stringData:        gcp.json: <CREDENTIALS_JSON_BLOB>
Or use 
kubectl
to create the secret directly from a file:
Copy code
bash    kubectl create secret generic gcp-cred-secrets --from-file=gcp.json=<path-to-your-credentials-file>.json
2. Update Airbyte Configuration In your 
airbyte.yml
configuration file, add the following configuration, adjusting 
<project-id>
to match your GCP project ID:
Copy code
yaml    global:        storage:          type: gcs          storageSecretName: gcp-cred-secrets          bucket:            log: airbyte-storage            state: airbyte-storage            workloadOutput: airbyte-storage          gcs:            authenticationType: credentials            projectId: <project-id>            credentialsPath: /secrets/gcs-log-creds/gcp.json              secretsManager:          type: googleSecretManager          storageSecretName: gcp-cred-secrets          googleSecretManager:            authenticationType: credentials            projectId: <project-id>            credentialsSecretKey: gcp-creds.json
3. Remove Deprecated Configuration from 
values.yaml
 Edit your 
values.yaml
files to remove any deprecated storage and secrets manager environment variables related to GCS. Ensure configurations like 
global.state.storage.type: "GCS"
and GCS credentials paths under 
extraEnv
are removed. Please note that this guide ensures that you leverage the latest Helm chart configurations for Airbyte, aligning with best practices for managing storage and secrets in Kubernetes environments for AWS and GCS users. For more details, you can refer to the [official documentation](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm#migration-steps-2).
Open in Slack
PreviousNext
Powered by