https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT>, is this the correct way of adding ...
# replication-ask-ai
d
@kapa.ai, is this the correct way of adding a tolerations for the different deployments? temporal: nodeSelector: airbyte_node_type: core tolerations: - key: airbyte_node_type operator: Equal value: core effect: NoSchedule
k
Yes, that is the correct format for adding tolerations to a specific deployment like 
temporal
in Airbyte's Helm chart. The structure you're using matches the standard Kubernetes toleration spec and is consistent with examples found in Airbyte's configuration. [Helm chart values] For example, a similar pattern is used in practice:
Copy code
tolerations:
  - key: abc/node-group
    value: airbyte-workers
    effect: NoSchedule
    operator: Equal
[EKS issue] Your configuration looks correct:
Copy code
temporal:
  nodeSelector:
    airbyte_node_type: core
  tolerations:
    - key: airbyte_node_type
      operator: Equal
      value: core
      effect: NoSchedule
A few things to keep in mind: • The taint must exist on the node — the toleration only allows the pod to be scheduled on a node with a matching taint. Make sure the node has the taint 
airbyte_node_type=core:NoSchedule
applied. • 
nodeSelector
+ 
tolerations
work together — the 
nodeSelector
ensures the pod is attracted to the right node, while the 
toleration
ensures it's allowed on a tainted node. • This pattern applies per-component (e.g., 
temporal
, 
server
, 
worker
, etc.), and you can apply similar blocks to other deployments as needed. [Scaling Airbyte] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, I've added this block to all airbyte components, like temporal, server, cron etc., but only temporal and airbyte-connector-builder-server deployments, doesn't have that toleration applied despite the fact that added and applied this at the same time for all components. what could be the reason?
k
This is a known issue with Airbyte's Helm Chart V2. There is an open GitHub issue that directly explains what you're experiencing. The 
temporal
and 
connector-builder-server
(manifest-server) deployment templates are missing the `global.extraVolumes`/`global.extraVolumeMounts` blocks, but more relevantly to your case, the audit found that several deployment templates do not correctly implement all global configurations. [Helm Chart V2 issue] Specifically, the affected templates are: The correctly implemented templates include 
airbyte-worker
, 
airbyte-workload-launcher
, 
airbyte-workload-api-server
, and 
airbyte-webapp
. [Helm Chart V2 issue] Workaround The recommended workaround is to patch the affected templates locally to add the missing blocks. For each affected deployment template's 
volumes:
section, add:
Copy code
{{- if .Values.global.extraVolumes }}
{{ toYaml .Values.global.extraVolumes | nindent 6 }}
{{- end }}
And for 
airbyte-temporal/deployment.yaml
, also add to the container's 
volumeMounts:
section:
Copy code
{{- if .Values.global.extraVolumeMounts }}
  {{ toYaml .Values.global.extraVolumeMounts | nindent 8 }}
{{- end }}
This issue has been escalated to Airbyte's internal engineering team, but as of the latest information available, it has not yet been resolved in the chart. [Helm Chart V2 issue] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
d
@kapa.ai, what the extraVolums has to do with the tolerations block?
k
You're absolutely right to call that out — I apologize for the confusion. The 
extraVolumes
issue is a separate bug and has nothing to do with tolerations not being applied. The knowledge sources I have available do not contain enough specific information to explain definitively why tolerations set at the component level (e.g., 
temporal.tolerations
, 
connector-builder-server.tolerations
) would be silently ignored for those specific deployments while working for others. However, some possible things worth investigating: 1. Helm chart template bugs — Similar to the 
extraVolumes
issue, it's possible that the 
temporal
and 
connector-builder-server
deployment templates simply don't correctly wire the component-level 
tolerations
values into the pod spec. This would be a Helm chart template bug. 2. Check the rendered templates — You can verify what Helm actually generates for those deployments by running:helm template airbyte airbyte/airbyte -f your-values.yaml \ | grep -A 20 "kind: Deployment" Then check whether the 
tolerations
block appears in the rendered output for 
temporal
and 
connector-builder-server
. 3. Report it — If the tolerations are missing from the rendered templates, it would be worth opening a GitHub issue similar to the 
extraVolumes
one at github.com/airbytehq/airbyte/issues, as it follows the same pattern of incomplete template implementation. I'd recommend verifying with 
helm template
first to confirm whether this is indeed a chart template bug. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, none of the components, like event cron, server etc. do not have tolerations block, so not sure if this is correct test
k
You're right, and I want to be transparent here — the knowledge sources I have available do not contain enough information to definitively explain why tolerations are not being applied to specific components like 
temporal
and 
connector-builder-server
in your setup. What I can confirm from the knowledge sources is that the correct format for component-level tolerations in the Helm chart is: [Helm chart values]
Copy code
temporal:
  tolerations:
    - key: airbyte_node_type
      operator: Equal
      value: core
      effect: NoSchedule
And that this pattern has been used successfully for components like 
worker
and 
workload-api-server
. [EKS issue] My honest recommendation would be to: 1. Open a GitHub issue at github.com/airbytehq/airbyte/issues describing which components are missing the tolerations, similar to how the 
extraVolumes
bug was reported and confirmed. 2. Reach out on the Airbyte community Slack where Airbyte engineers are active and may have direct knowledge of this. I don't want to speculate further without reliable information to back it up. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, can I redeploy somehow the helm chart to check if these tolerations will be applied? Note that I use TF for this
k
The knowledge sources I have available do not contain specific information about using Terraform to deploy/redeploy Airbyte Helm charts. However, based on what the knowledge sources do show, the standard way to redeploy with Helm is using `helm upgrade`: [Upgrading Airbyte]
Copy code
helm upgrade airbyte airbyte/airbyte \
  --namespace airbyte \
  --values ./values.yaml \
  --version 2.x.x
If you're using Terraform with the Helm provider, the equivalent would typically be triggering a 
terraform apply
after updating your values — but I cannot confirm the exact Terraform configuration details from the knowledge sources available. What I'd suggest to verify tolerations before a full redeploy: Use 
helm template
to dry-run and inspect the rendered manifests without actually deploying:
Copy code
helm template airbyte airbyte/airbyte \
  --namespace airbyte \
  --values ./values.yaml \
  --version 2.x.x \
  | grep -A 10 "tolerations"
This will let you confirm whether the tolerations are present in the rendered output for all components, including 
temporal
and 
connector-builder-server
, without touching your live environment. This is a safe first step regardless of whether you use Terraform or plain Helm. For Terraform-specific guidance, I'd recommend checking the Terraform Helm provider docs directly. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, where should the temporal be, under global - global.temporal, or only temporal like server ?
k
Based on the knowledge sources available, I can see examples of 
temporal
being configured at the top level (same level as 
server
, 
worker
, etc.), not under `global`:
Copy code
temporal:
  replicaCount: 1
  tolerations: []
  nodeSelector: {}
  resources:
    requests: {}
    limits: {}
  extraEnv: []
[Helm chart values] And from a real-world deployment example:
Copy code
temporal:
  resources:
    requests:
      memory: 100Mi
      cpu: 50m
    limits:
      memory: 512Mi
      cpu: 200m
  extraEnv:
    - name: SKIP_DB_CREATE
      value: "true"
[EKS issue] However, there is a separate 
global.temporal
namespace used for specific settings like Temporal Cloud configuration: [Chart V2 upgrade] So in summary: • Component-level settings (tolerations, nodeSelector, resources, replicas) → 
temporal.*
• Temporal Cloud/CLI settings → 
global.temporal.*
👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
d
@kapa.ai, and what about the connectorBuilderServer ?
k
Based on the knowledge sources, 
connectorBuilderServer
has gone through a rename depending on which Helm chart version you're using: Helm Chart V1 — use `connectorBuilderServer`:
Copy code
connectorBuilderServer:
  replicaCount: 1
  resources:
    requests:
      memory: 256Mi
      cpu: 200m
    limits:
      memory: 1Gi
      cpu: 500m
[Helm chart V1] Helm Chart V2 — the 
connector-builder-server
was renamed to 
manifestServer
:
Copy code
manifestServer:
  extraEnv:
    - name: AIRBYTE_ENABLE_UNSAFE_CODE
      value: "true"
[Chart V2 fix] So if you're on Helm Chart V2 (which current 
abctl
versions install), you should use 
manifestServer
for component-level settings like tolerations and nodeSelector, not 
connectorBuilderServer
— as Helm will silently ignore the old key without any error. [Chart V2 fix] There is also a 
global.connectorBuilderServer
namespace used for specific API settings like: [Chart V2 upgrade] So in summary: • Component-level settings (tolerations, nodeSelector, resources) → 
manifestServer.*
(Chart V2) • API host settings → 
global.connectorBuilderServer.*
👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
d
@kapa.ai, I'm using helm v2 but airbyte 2.0.19, and I see that 
manifestServer
was introduced in 2.1
k
You raise a good point. Based on the knowledge sources, the release notes confirm that the 
connector-builder-server
component was replaced by 
manifest-server
in Airbyte v2.1.0: [v2.1 release notes] > The 
connector-builder-server
component is removed from the Helm chart and replaced by the manifest server. If you reference 
connector-builder-server
in values.yaml, update those references. So for your setup (Helm Chart V2 + Airbyte 2.0.19), you should still be using `connectorBuilderServer`:
Copy code
connectorBuilderServer:
  tolerations:
    - key: airbyte_node_type
      operator: Equal
      value: core
      effect: NoSchedule
  nodeSelector:
    airbyte_node_type: core
And 
manifestServer
would only apply if you upgrade to Airbyte 2.1.0 or later. [v2.1 release notes] That said, the knowledge sources do not contain enough detail to confirm whether 
connectorBuilderServer.tolerations
is correctly wired in the Helm Chart V2 templates for Airbyte 2.0.x. I'd recommend using 
helm template
to verify the rendered output as suggested earlier. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, so is connectorBuilderServer under global or it is configured at the top level ?
k
Based on the knowledge sources, 
connectorBuilderServer
exists in both places but for different purposes: Top-level — for component-level settings (tolerations, nodeSelector, resources, etc.):
Copy code
connectorBuilderServer:
  tolerations:
    - key: airbyte_node_type
      operator: Equal
      value: core
      effect: NoSchedule
  nodeSelector:
    airbyte_node_type: core
[Helm chart V1] Under 
global
 — only for specific API host settings: [Chart V2 upgrade] So for your use case of adding tolerations, it should be at the top level under 
connectorBuilderServer
, not under 
global
. That said, I want to be transparent — the knowledge sources don't explicitly confirm whether 
connectorBuilderServer.tolerations
is correctly wired in the Helm chart templates for Airbyte 2.0.19. I'd still recommend verifying with 
helm template
to check the rendered output. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
d
@kapa.ai, what are the expected values for secret airbyte-config-secrets ?
k
Based on the knowledge sources, the expected keys in the 
airbyte-config-secrets
Kubernetes secret depend on which integrations you are using: Core / Enterprise:
Copy code
stringData:
  # Enterprise License Key
  license-key: ## e.g. xxxxx.yyyyy.zzzzz

  # Database Secrets
  database-host: ## e.g. database.internal
  database-port: ## e.g. 5432
  database-name: ## e.g. airbyte
  database-user: ## e.g. airbyte
  database-password: ## e.g. password

  # Instance Admin
  instance-admin-email: ## e.g. admin@example.com
  instance-admin-password: ## e.g. password

  # SSO OIDC Credentials
  client-id: ## e.g. e83bbc57-1991-417f-8203-3affb47636cf
  client-secret: ## e.g. wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[Enterprise setup] S3 Storage:
Copy code
stringData:
  s3-access-key-id: ## e.g. AKIAIOSFODNN7EXAMPLE
  s3-secret-access-key: ## e.g. wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[S3 storage] GCS Storage:
Copy code
stringData:
  gcp.json: <CREDENTIALS_JSON_BLOB>
[GCS storage] Azure Blob Storage:
Copy code
stringData:
  azure-blob-store-connection-string: ## DefaultEndpointsProtocol=https;...
[Azure storage] AWS Secrets Manager:
Copy code
stringData:
  aws-secret-manager-access-key-id: ## e.g. AKIAIOSFODNN7EXAMPLE
  aws-secret-manager-secret-access-key: ## e.g. wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[AWS Secrets Manager] Azure Key Vault:
Copy code
stringData:
  azure-key-vault-client-id: ## e.g. 3fc863e9-4740-4871-bdd4-456903a04d4e
  azure-key-vault-client-secret: ## e.g. KWP6egqixiQeQoKqFZuZq2weRbYoVxMH
[Azure Key Vault] Key points: • You only need to include the keys relevant to your specific integrations — not all of them are required. • If you're using 
instanceProfile
authentication for AWS (IAM roles), you don't need to include the AWS access key entries. • All of these can live in the same single secret (
airbyte-config-secrets
). [Preconfiguring secrets] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, my question is related to the secret "airbyte-config-secrets". should it have database-host: database-name: database-port: or only database-password: database-user: is enough ?
k
Based on the knowledge sources, there are two different approaches and it depends on how you configure your `values.yaml`: Option 1 — Minimum (host/port/name in 
values.yaml
directly): If you specify 
host
, 
port
, and 
name
directly in your 
values.yaml
, then the secret only needs:
Copy code
stringData:
  database-user: ## e.g. airbyte
  database-password: ## e.g. password
[External database] Option 2 — Full credentials in the secret: If you prefer to store everything in the secret, you can include all database details:
Copy code
stringData:
  database-host: ## e.g. database.internal
  database-port: ## e.g. 5432
  database-name: ## e.g. airbyte
  database-user: ## e.g. airbyte
  database-password: ## e.g. password
[Enterprise setup] The key distinction is explicitly stated in the docs: > You can override all of the values below by setting them in the 
airbyte-config-secrets
or set them directly in the 
values.yaml
file. The database password is a special case in that it must be set in the 
airbyte-config-secrets
. [External database] So in summary: • 
database-password
→ must always be in the secret • 
database-user
→ can be in the secret or directly in 
values.yaml
• 
database-host
, 
database-port
, 
database-name
→ optional in the secret, can be set directly in 
values.yaml
👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, i have instance-admin-password and instance-admin-email in airbyte-config-secrets, but they do not work to login into airbyte
k
This is a known issue documented in the knowledge sources. The problem is that 
airbyte-auth-secrets
 and 
airbyte-config-secrets
 are two different secrets used for different purposes. For login credentials specifically, Airbyte expects 
instance-admin-password
and 
instance-admin-email
to be in 
airbyte-auth-secrets
, not in 
airbyte-config-secrets
. [Authentication docs] Create or update the 
airbyte-auth-secrets
secret:
Copy code
apiVersion: v1
kind: Secret
metadata:
  name: airbyte-auth-secrets
type: Opaque
stringData:
  instance-admin-password: ## your password
  instance-admin-email: ## your email
Or via `kubectl`:
Copy code
kubectl create secret generic airbyte-auth-secrets \
  --from-literal=instance-admin-password='' \
  --from-literal=instance-admin-email='' \
  --namespace airbyte
There is also a known Helm chart bug where even if you configure 
global.auth.instanceAdmin.secretName
to point to a custom secret, the bootloader hardcodes the secret name to 
airbyte-auth-secrets
and ignores your custom secret name. [Helm chart bug] So the workaround is to either: 1. Put credentials in 
airbyte-auth-secrets
 (recommended, works reliably) [Authentication docs] 2. Override via 
extraEnv
 in your 
values.yaml
for the bootloader:
Copy code
airbyte-bootloader:
  extraEnv:
    - name: AB_INSTANCE_ADMIN_PASSWORD
      valueFrom:
        secretKeyRef:
          name: "airbyte-config-secrets"
          key: "instance-admin-password"
    - name: AB_INSTANCE_ADMIN_EMAIL
      valueFrom:
        secretKeyRef:
          name: "airbyte-config-secrets"
          key: "instance-admin-email"
[Helm chart bug] After applying the changes, you may need to restart the 
airbyte-server
pod for the changes to take effect. [Authentication docs] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, 
airbyte-auth-secrets
 is created by airbyte itself, right? If that's the case how can I use terraform and external secret to manage this secret without affecting the values for jtw-signature-secret, dataplane-client-id and dataplance-client-secret ?
k
This is a great question, and the knowledge sources do confirm parts of it but not all. Yes, 
airbyte-auth-secrets
is created and managed by Airbyte itself (specifically the bootloader), and it stores multiple keys: [Helm chart bug] • 
instance-admin-password
• 
instance-admin-email
• 
jwt-signature-secret
• 
dataplane-client-id
• 
dataplane-client-secret
The knowledge sources confirm this is exactly the problem many users face — if you manage 
airbyte-auth-secrets
externally, the bootloader will overwrite 
dataplane-client-id
and 
dataplane-client-secret
on every redeploy. [Bootloader issue] A fix was introduced in Airbyte 1.8.0 where the bootloader now checks for valid existing credentials first before overwriting: [Bootloader fix] > The new behavior will look for valid credentials in the secret first, and if valid credentials exist, it will skip the rest of the dataplane initialization. However, the knowledge sources do not contain enough information to confirm a fully reliable Terraform + External Secrets approach for managing 
airbyte-auth-secrets
without side effects. What users have found to work as a workaround is: [v1.6 issue] 1. Let Airbyte create 
airbyte-auth-secrets
on first deploy 2. Extract the generated values for 
jwt-signature-secret
, 
dataplane-client-id
, and 
dataplane-client-secret
3. Include all keys together in your externally managed secret
Copy code
stringData:
  instance-admin-password: ## your password
  instance-admin-email: ## your email
  jwt-signature-secret: ## value from initial deploy
  dataplane-client-id: ## value from initial deploy
  dataplane-client-secret: ## value from initial deploy
For the specific Terraform + External Secrets implementation details, I'd recommend checking the Airbyte community Slack as the knowledge sources don't cover this combination in enough detail to give a confident answer. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, 
airbyte-auth-secrets
is created by airbyte, but using the values in it I cannot login into airbyte
k
Based on the knowledge sources, here are the confirmed reasons why you might not be able to login using the values in `airbyte-auth-secrets`: 1. The password is base64 encoded The values in 
airbyte-auth-secrets
are base64 encoded, so you need to decode them first to get the actual password: [Authentication docs]
Copy code
kubectl get secret airbyte-auth-secrets -n <YOUR_NAMESPACE> -o yaml
Then decode the password:
Copy code
echo '<base64-encoded-value>' | base64 -d
2. The bootloader may have used empty values There is a known bug where the bootloader sets the password as an empty value and generates its own, ignoring the values you provided in the secret: [Helm chart bug] > The bootloader (which creates the initial admin user) sets the password as an empty value 3. The email may not be set correctly The knowledge sources confirm that the email is stored in the 
organization
table in the database, not necessarily driven by the secret. One user confirmed: [GitHub discussion] > I updated the field 
email
in the table 
organization
in the db. The password is indeed 
instance-admin-password
in the secret 
airbyte-auth-secrets
. Suggested steps: 1. Check the actual decoded value of 
instance-admin-password
in `airbyte-auth-secrets`:
Copy code
kubectl get secret airbyte-auth-secrets -n <YOUR_NAMESPACE> -o jsonpath='{.data.instance-admin-password}' | base64 -d
2. Try logging in with that decoded password. 3. If the password is empty or incorrect, apply the 
extraEnv
workaround for the bootloader to force it to use your desired credentials. [Helm chart bug] Beyond these points, the knowledge sources do not contain enough information to diagnose further why login is failing in your specific case. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, I know it's base64, but the questions is what to use as user for login?
k
Based on the knowledge sources, the email is used as the username to login into Airbyte. You can find what email is set by running: [Authentication docs]
Copy code
kubectl get secret airbyte-auth-secrets -n <YOUR_NAMESPACE> -o jsonpath='{.data.instance-admin-email}' | base64 -d
However, there is a known issue where the email may show as 
[not set]
 even after deployment: [Email not set issue]
Copy code
Email: [not set]
Password: B5GR3LtbbcNTdIWigxxhf9M3HkGlIZ0i
If that's the case, the confirmed workaround is to directly update the 
email
field in the 
organization
table in the Airbyte database: [GitHub discussion] > I updated the field 
email
in the table 
organization
in the db. The password is indeed 
instance-admin-password
in the secret 
airbyte-auth-secrets
.
Copy code
UPDATE organization SET email = 'your@email.com' WHERE name = 'Default Organization';
So in summary: • Username → the email stored in 
airbyte-auth-secrets
under 
instance-admin-email
, or the email in the 
organization
table in the database • Password → the decoded value of 
instance-admin-password
in 
airbyte-auth-secrets
👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, can I have 2 airbyte intallations having the same redshift destination?
k
Based on the knowledge sources, technically yes you can have 2 Airbyte installations sharing the same Redshift destination, but with important restrictions. As of Airbyte 1.6, unique stream identifiers are mandatory across connections sharing the same destination: [Unique stream prefixes] > Airbyte now requires that stream identifiers be unique across Airbyte connections that share a destination. You can no longer set multiple connections to sync to the same destination stream. To avoid conflicts between the two installations, you must ensure uniqueness by either: Option 1 — Unique namespace per installation:
Copy code
# Installation 1
namespace: airbyte_install_1

# Installation 2
namespace: airbyte_install_2
Option 2 — Unique stream prefix per installation:
Copy code
# Installation 1
streamPrefix: install1_

# Installation 2
streamPrefix: install2_
[Configuring schemas] Since Redshift maps namespaces to schemas, each installation writing to a unique namespace will effectively write to a separate Redshift schema, avoiding any conflicts. [Redshift namespace] Key risks if you don't ensure uniqueness: • Job failures • Data loss due to multiple connections clashing while writing to the same table [Configuring schemas] If you want to combine data from both installations into a single table, the recommended approach is to do that downstream in Redshift after Airbyte has synced the data, rather than syncing directly to the same table. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai, how can I used the pod identity eks addon in airbyte?
k
Based on the knowledge sources, this is a known open issue — EKS Pod Identity is currently not fully supported by Airbyte's Java-based connectors due to an AWS SDK v1 limitation. [Pod Identity issue] Root Cause EKS Pod Identity injects two environment variables: • 
AWS_CONTAINER_CREDENTIALS_FULL_URI
• 
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE
← this is the problem AWS Java SDK v1 only handles 
AWS_CONTAINER_AUTHORIZATION_TOKEN
(literal token value), not the file-based variant (
_FILE
). As a result, Java connectors fall through the credential provider chain and fail with:
Copy code
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain:
  [ContainerCredentialsProvider: Failed to connect to container credentials endpoint: 401 Unauthorized]
[Pod Identity issue] Affected Components • 
source-s3
, 
destination-s3
• 
source-dynamodb
• 
destination-redshift
, 
source-redshift
• Any other Java-based connector using AWS SDK What Works vs What Doesn't Current Workaround — Use IRSA instead Since IRSA uses a different injection mechanism that AWS Java SDK v1 does support, switching from Pod Identity to IRSA resolves the issue: [Pod Identity issue]
Copy code
serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<your-role>
And in your `values.yaml`:
Copy code
global:
  storage:
    type: S3
    s3:
      region: <your-region>
      authenticationType: instanceProfile
Future Fix There is an open roadmap item to support EKS Pod Identity natively across all Airbyte deployment models, which would require upgrading affected connectors to AWS Java SDK v2. [AWS credential roadmap] Until that is resolved, IRSA remains the recommended authentication method for EKS deployments. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
d
@kapa.ai, how can I add annotations to the airbyte-admin serviceAccount ?
k
Based on the knowledge sources, you can add annotations to the service account via your 
values.yaml
using the 
serviceAccount.annotations
field at the top level:
Copy code
serviceAccount:
  create: true
  name: airbyte-admin
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<your-role>
[Helm chart values] This is the pattern confirmed by multiple users deploying Airbyte on EKS with IRSA. [IRSA issue] However, there is a known issue where using a custom service account name (other than 
airbyte-admin
) causes problems with replication pods. So if you're keeping the default 
airbyte-admin
name and just adding annotations, this should work fine. [Service account issue] Important caveat — there is also a known bug where 
serviceAccount.name
alone is not enough to override the default, and you may also need to set 
global.serviceAccountName
to match: [Override issue]
Copy code
global:
  serviceAccountName: airbyte-admin

serviceAccount:
  create: true
  name: airbyte-admin
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<your-role>
This ensures both the service account creation and its usage across all components are consistent. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
7 Views
Open in Slack
PreviousNext
Powered by