https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT>: Give me complete helm chart exampl...
# replication-ask-ai
k
@kapa.ai: Give me complete helm chart example to deploy airbyte 2.0.18 on Azure AKS with secrets reading from Azure Key vault and logs integrated to Azure blob storage containers.
k
Here is a complete example for deploying Airbyte 2.0 (Helm chart version 2.0.18) on Azure AKS with Azure Key Vault for secret management and Azure Blob Storage for logs. 1. Add the Helm Repository
Copy code
helm repo add airbyte https://airbytehq.github.io/charts
helm repo update
[Installation Guide] 2. Create the Kubernetes Namespace
Copy code
kubectl create namespace airbyte
3. Create Kubernetes Secrets You need two secrets: one for Azure Blob Storage and one for Azure Key Vault. 
airbyte-config-secrets.yaml
Copy code
apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
type: Opaque
stringData:
  # Azure Blob Storage connection string
  azure-blob-store-connection-string: "DefaultEndpointsProtocol=https;AccountName=mystorageaccount;AccountKey=<YOUR_ACCOUNT_KEY>;EndpointSuffix=core.windows.net"

  # Azure Key Vault service principal credentials
  azure-key-vault-client-id: "<your-client-id>"         # e.g. 3fc863e9-4740-4871-bdd4-456903a04d4e
  azure-key-vault-client-secret: "<your-client-secret>" # e.g. KWP6egqixiQeQoKqFZuZq2weRbYoVxMH
Apply it:
Copy code
kubectl apply -f airbyte-config-secrets.yaml -n airbyte
[Storage Secrets; Secret Management] 4. Create values.yaml
Copy code
global:
  edition: community
  airbyteUrl: "https://airbyte.your-domain.example"  # Replace with your actual URL

  # External Azure Blob Storage for logs and state
  storage:
    type: "Azure"
    secretName: airbyte-config-secrets
    bucket:
      log: airbyte-container          # Name of your Azure Blob container
      state: airbyte-container
      workloadOutput: airbyte-container
      activityPayload: airbyte-container
    azure:
      connectionStringSecretKey: azure-blob-store-connection-string  # Key in your Kubernetes secret

  # Azure Key Vault for connector secret management
  secretsManager:
    type: AZURE_KEY_VAULT
    secretsManagerSecretName: "airbyte-config-secrets"
    azureKeyVault:
      vaultUrl: "https://my-vault.vault.azure.net/"       # Replace with your Key Vault URL
      tenantId: "your-tenant-id"                          # Replace with your Azure Tenant ID
      clientId: ""                                        # Inline value (optional if using secretKey)
      clientIdSecretKey: "azure-key-vault-client-id"      # Key in your Kubernetes secret
      clientSecret: ""                                    # Inline value (optional if using secretKey)
      clientSecretSecretKey: "azure-key-vault-client-secret" # Key in your Kubernetes secret

  # External PostgreSQL database (recommended for production)
  database:
    secretName: "airbyte-config-secrets"
    host: "your-postgres-host"
    port: 5432
    name: "airbyte"
    userSecretKey: "database-user"
    passwordSecretKey: "database-password"

postgresql:
  enabled: false  # Disable internal Postgres since we use external

# Minio is disabled because we use Azure Blob Storage
minio:
  enabled: false
> Note: The 
database
section is recommended for production AKS deployments. If you are testing without an external database, remove the 
database
block and set 
postgresql.enabled: true
. Also add 
database-user
and 
database-password
keys to your Kubernetes secret if using an external DB. [Storage Values; Secret Management Values; Chart V2 values.yaml] 5. Install Airbyte
Copy code
helm install airbyte airbyte/airbyte \
  --namespace airbyte \
  --values ./values.yaml \
  --version 2.0.18
[Installing Airbyte] Key Notes • Adding external Azure Blob Storage in 
values.yaml
automatically disables the default internal Minio instance. [Storage Secrets] • Airbyte recommends using a single blob container across all three bucket values (
log
, 
state
, 
workloadOutput
). [Storage Values] • Modifying the connector secret storage configuration will cause existing connectors to fail — you will need to recreate them. [Secret Management Values] • For production AKS, Airbyte recommends deploying to Azure Kubernetes Service running in 2 or more availability zones. [Infrastructure Prerequisites] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
I can identify several issues in your 
values.yaml
based on the knowledge sources, but I must be upfront that some observations go beyond what the knowledge sources explicitly cover. Issues Found 1. global.airbyteUrl — Malformed URL Your current value uses a Slack-formatted hyperlink:
Copy code
airbyteUrl: <https://airbyte-v2-test.com|airbyte-v2-test.com>
It should be a plain URL string:
Copy code
airbyteUrl: "https://airbyte-v2-test.com"
[Chart V2 values.yaml] 2. global.database.host — Malformed URL Same Slack hyperlink formatting issue:
Copy code
host: "<http://postgres.database.azure.com|postgres.database.azure.com>"
Should be:
Copy code
host: "postgres.database.azure.com"
Also note: the host should not include 
http://
— it should be a plain hostname. [Chart V2 values.yaml] 3. global.database.port — Should Be an Integer, Not a String
Copy code
port: "5432"   # ❌ string
Should be:
Copy code
port: 5432     # ✅ integer
[Chart V2 values.yaml] 4. global.database — Missing secretName, userSecretKey, passwordSecretKey Your database block is missing credentials configuration entirely. You need either inline credentials or secret references:
Copy code
global:
  database:
    host: "postgres.database.azure.com"
    port: 5432
    name: "airbyte-v2-dv"
    secretName: "airbyte-config-secrets"
    userSecretKey: "database-user"
    passwordSecretKey: "database-password"
[Chart V2 values.yaml] 5. ingress.hosts — Malformed Host URLs Same Slack hyperlink formatting issue on the ingress hosts:
Copy code
host: <http://airbyte-v2-test.com|airbyte-v2-test.com>  # ❌
Should be:
Copy code
host: airbyte-v2-test.com  # ✅ plain hostname, no protocol
Also, TLS hosts should not include 
http://
. [Ingress] 6. ingress — Missing className The ingress block is missing the 
className
field, which is required to specify your ingress controller (e.g., 
nginx
, 
alb
):
Copy code
ingress:
  enabled: true
  className: "nginx"   # ← Add this
[Ingress] 7. global.airbyteUrl Must Match Ingress Host The docs explicitly require these to match:
Copy code
global:
  airbyteUrl: "https://airbyte-v2-test.com"

ingress:
  hosts:
    - host: airbyte-v2-test.com
[Ingress URL Match] Items I Cannot Confirm from Knowledge Sources The knowledge sources do not explicitly document the following fields you are using, so I cannot confirm their correctness: • 
global.dataplaneGroups.defaultDataplaneGroupName
• 
WORKLOAD_LAUNCHER_ENABLED_NAMESPACES
• 
global.logging.level
These may be valid, but I'd recommend verifying them against the Values.yaml reference directly. Summary of Fixes 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
I've reviewed your 
values.yaml
. Here are my observations, organized by category: ✅ Things That Look Correct • 
global.jobs.kube.namespace
 — Correctly scoped to your namespace. • 
global.airbyteUrl
 — Present and set. • 
workloadLauncher
/ 
worker
/ 
workloadApiServer
auth alignment — Sharing the same bearer token via 
secretKeyRef
is the right approach to fix the 401 auth issue described in GitHub issue #70451. [Auth Issue] • 
ingress.enabled: true
 with 
backend: server
and 
backend: connector-builder-server
paths — Correct per Helm chart V2 ingress requirements. [Ingress Config] ⚠️ Issues and Recommendations 1. global.airbyteUrl format looks malformed Your value appears as:
Copy code
airbyteUrl: <https://airbyte-v2-test.com|airbyte-v2-test.com>
This looks like a Slack-formatted hyperlink that was pasted in. It should be a plain URL:
Copy code
airbyteUrl: "https://airbyte-v2-test.com"
2. global.database.host format looks malformed Similarly:
Copy code
host: "<http://postgres.database.azure.com|postgres.database.azure.com>"
Should be:
Copy code
host: "postgres.database.azure.com"
Also note: Azure PostgreSQL Flexible Server hostnames use HTTPS under the hood, but the 
host
value should be the plain hostname only, not a URL with a scheme. [Database Config] 3. global.database.port should be an integer, not a string
Copy code
# ❌ Current
port: "5432"

# ✅ Correct
port: 5432
[Database Config] 4. Missing global.database credentials Your database block has 
host
, 
name
, and 
port
, but no credentials:
Copy code
global:
  database:
    host: "postgres.database.azure.com"
    name: "airbyte-v2-dv"
    port: 5432
    secretName: "airbyte-config-secrets"   # ← ADD
    userSecretKey: "database-user"          # ← ADD
    passwordSecretKey: "database-password"  # ← ADD
Without credentials, the bootloader and server will fail to connect to the database. [Database Config] 5. postgresql.enabled is not explicitly set to false Since you're using an external database, you should explicitly disable the bundled PostgreSQL:
Copy code
postgresql:
  enabled: false
[External DB] 6. Ingress host format looks malformed
Copy code
# ❌ Current (Slack-formatted)
host: <http://airbyte-v2-test.com|airbyte-v2-test.com>

# ✅ Correct
host: airbyte-v2-test.com
Also, TLS hosts should use the plain hostname (not a URL with scheme). [Ingress Config] 7. global.airbyteUrl must match the ingress host exactly Per Airbyte's requirement:
Copy code
global:
  airbyteUrl: "https://airbyte-v2-test.com"  # Must match ingress host
[Ingress URL Match] 8. Missing ingress.className The ingress block is missing a 
className
, which is required for most Kubernetes ingress controllers (e.g., NGINX on AKS):
Copy code
ingress:
  enabled: true
  className: "nginx"   # ← ADD, or "azure/application-gateway" etc.
[Ingress Config] ℹ️ Items I Cannot Fully Validate The knowledge sources do not contain explicit documentation on the following keys you've used, so I cannot confirm their exact correctness: • 
global.dataplaneGroups.defaultDataplaneGroupName
• 
WORKLOAD_LAUNCHER_ENABLED_NAMESPACES
These may be valid internal environment variables, but they are not documented in the available Airbyte Helm chart V2 reference material I have access to. I'd recommend verifying these against the official values.yaml reference. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
I'll review your global config file in the context of the environment-specific values file reviewed earlier. Here's my analysis: ✅ Things That Look Correct • 
postgresql.enabled: false
 and 
minio.enabled: false
 — Correctly disabled since you're using external DB and Azure Blob Storage. [Chart V2] • 
global.database.type: "external"
 — Correct key for external DB. [External DB] • 
global.database.port: 5432
 — Correctly an integer (unlike the env overlay which had it as a string). • 
global.storage.type: azure
 with 
connectionStringSecretKey
— Correct pattern for Azure Blob Storage. [Storage Values] • 
global.auth.security.secretName
and 
jwtSignatureSecretKey
 — Correct structure for JWT secret reference. • 
keycloak.enabled: false
 and 
keycloakSetup.enabled: false
 — Correct for community edition without SSO. • 
ingress.className: "nginx-internal"
 — Good, this was missing in the env overlay. • 
global.jobs.kube.mainContainerImagePullSecret
 — Correct key per Helm V2 spec. [Chart V2] ⚠️ Issues and Recommendations 1. global.airbyteUrl is Empty in Global Config
Copy code
airbyteUrl: ""  # Set in environment-specific overlay
The env overlay also has a malformed Slack-formatted URL. Make sure the overlay correctly sets this to a plain URL:
Copy code
airbyteUrl: "https://airbyte-v2-test.com"
This must match the ingress host exactly. [Chart V2] 2. global.database.host and global.database.name Are Empty
Copy code
host: ""  # Set in environment-specific overlay
name: ""  # Set in environment-specific overlay
The env overlay sets 
host
with a malformed Slack URL and is missing 
secretName
, 
userSecretKey
, and 
passwordSecretKey
. The global config does define these correctly:
Copy code
secretName: "airbyte-auth-secrets"
userSecretKey: "DATABASE_USER"
passwordSecretKey: "DATABASE_PASSWORD"
However, verify that the keys 
DATABASE_USER
and 
DATABASE_PASSWORD
exactly match the keys in your 
airbyte-auth-secrets
Kubernetes secret. The Airbyte docs use lowercase keys like 
database-user
and 
database-password
as examples — your uppercase naming is fine as long as it matches what's in the secret. [External DB] 3. global.logging.level Conflict Between Files • Global config sets: 
level: "INFO"
• Env overlay sets: 
level: "DEBUG"
The env overlay will win if it's applied as a Helm values override. This is likely intentional for the test environment, but worth being explicit about. [Chart V2] 4. global.storage.bucket.auditLogging Key You have:
Copy code
bucket:
  auditLogging: airbyte-v2-logs
Per the Helm V2 reference, the correct key mapping is:
Copy code
global.storage.bucket.auditLogging  →  STORAGE_BUCKET_AUDIT_LOGGING
This looks correct. [Chart V2] 5. global.auth.instanceAdmin — Missing secretName and passwordSecretKey You have the 
secretName
commented out:
Copy code
#secretName: "airbyte-auth-secrets"
#passwordSecretKey: "instance-admin-password"
firstName: "dfad"
lastName: "adfd"
email: novaplatformteam@gmail.com
The admin password must be provided either inline or via a secret reference. If you're relying on ExternalSecrets to inject 
instance-admin-password
into 
airbyte-auth-secrets
, you should uncomment these lines to ensure the chart can read the password:
Copy code
instanceAdmin:
  secretName: "airbyte-auth-secrets"
  passwordSecretKey: "instance-admin-password"
  firstName: "dfad"
  lastName: "adfd"
  email: "novaplatformteam@gmail.com"
[Chart V2] 6. global.edition Is Missing The global config does not set 
global.edition
. This is a required field in Helm chart V2:
Copy code
global:
  edition: community
[Chart V2] 7. workloadApiServer.service.port: 8001 The knowledge sources do not explicitly document this field. The default Airbyte server port is 
8001
, but I cannot confirm whether 
workloadApiServer.service.port
is a valid chart key — please verify against the values.yaml reference. 8. Env Overlay Still Missing Key Fixes When combined with the env overlay reviewed earlier, these issues from the overlay are not resolved by the global config: Summary The global config is well-structured overall, but has these critical gaps: 1. ❌ 
global.edition: community
is missing from both files 2. ❌ 
instanceAdmin.secretName
and 
passwordSecretKey
are commented out 3. ⚠️ Env overlay URL formatting issues must be fixed before deployment 4. ⚠️ Verify 
DATABASE_USER
/ 
DATABASE_PASSWORD
secret key casing matches your actual Kubernetes secret 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
Open in Slack
PreviousNext
Powered by