To set the affinity for the job pods launched by the worker in your EKS Airbyte OSS deployment, you need to use the

JOB_KUBE_NODE_SELECTORS

environment variable. This variable allows you to define node selectors for the job pods, ensuring they run on nodes with the specified architecture. Here’s how you can configure it: 1. Set the

JOB_KUBE_NODE_SELECTORS

Environment Variable: You can define the node selectors in your Helm chart or Kubernetes manifests using the

JOB_KUBE_NODE_SELECTORS

environment variable. This variable allows you to specify key-value pairs for node selection. 2. Example Configuration: Here is an example of how to set the

JOB_KUBE_NODE_SELECTORS

environment variable to ensure that job pods run on ARM nodes:

yaml env:   - name: JOB_KUBE_NODE_SELECTORS     value: "<http://kubernetes.io/arch=arm64|kubernetes.io/arch=arm64>"

3. Update Your Helm Chart or Kubernetes Manifests: Add the above environment variable configuration to your Helm chart or Kubernetes manifests. This will ensure that the job pods launched by the worker have the correct node selectors. 4. Apply the Configuration: After updating the Helm chart or Kubernetes manifests, apply the changes to your EKS cluster. 5. Verify the Configuration: Ensure that the job pods are running on ARM nodes by checking the node architecture of the running pods. Here is a summary of the relevant information from the knowledge sources: -

JOB_KUBE_NODE_SELECTORS

: Defines one or more Job pod node selectors. Each key-value pair is separated by a comma. For example:

key1=value1,key2=value2

. It is the pod node selectors of the sync job and the default pod node selectors fallback for other jobs ([source](https://docs.airbyte.com/operator-guides/configuring-airbyte#jobs-1)). By setting the

JOB_KUBE_NODE_SELECTORS

environment variable, you can ensure that the job pods launched by the worker run on nodes with the ARM architecture. This approach allows you to control the node selection for job pods without needing to modify the Helm chart directly for the

<http://global.jobs|global.jobs>

configuration. If you encounter further issues, you may need to check the specific images being used and ensure they are compatible with ARM architecture. You can also refer to the official Airbyte documentation for more detailed guidance on configuring node selectors and handling mixed architecture clusters.

To define multiple key-value pairs for the

JOB_KUBE_NODE_SELECTORS

variable within the

<http://global.jobs|global.jobs>

configuration of the Helm chart values, you can set the environment variable with the key-value pairs separated by commas. Here’s how you can do it: 1. Define the Environment Variable in the Helm Chart Values: You need to add the

JOB_KUBE_NODE_SELECTORS

environment variable under the

extraEnv

section in your Helm chart values file. 2. Example Configuration: Here is an example of how to set multiple key-value pairs for the

JOB_KUBE_NODE_SELECTORS

environment variable:

yaml worker:   extraEnv:     - name: JOB_KUBE_NODE_SELECTORS       value: "key1=value1,key2=value2"

In this example, replace

key1=value1,key2=value2

with your actual node selector key-value pairs. 3. Apply the Configuration: After updating the Helm chart values file, apply the changes to your EKS cluster using Helm. 4. Verify the Configuration: Ensure that the job pods are running on the nodes that match the specified node selectors by checking the node architecture of the running pods. Here is a summary of the relevant information from the knowledge sources: -

JOB_KUBE_NODE_SELECTORS

: Defines one or more Job pod node selectors. Each key-value pair is separated by a comma. For example:

key1=value1,key2=value2

. It is the pod node selectors of the sync job and the default pod node selectors fallback for other jobs ([source](https://docs.airbyte.com/operator-guides/configuring-airbyte#jobs-1)). By setting the

JOB_KUBE_NODE_SELECTORS

environment variable with multiple key-value pairs, you can ensure that the job pods launched by the worker run on nodes that match the specified criteria. If you encounter further issues, you may need to check the specific images being used and ensure they are compatible with the target architecture. You can also refer to the official Airbyte documentation for more detailed guidance on configuring node selectors and handling mixed architecture clusters.

To set the

JOB_KUBE_NODE_SELECTORS

using the

global.jobs.kube.nodeSelector

configuration in the latest Helm chart versions, you can define the node selectors directly in the

values.yaml

file under the

global.jobs.kube.nodeSelector

section. This approach ensures that the job pods launched by the worker will use the specified node selectors. Here’s an example of how to configure this in your

values.yaml

file:

yaml global:   jobs:     kube:       nodeSelector:         key1: value1         key2: value2

In this example, replace

key1: value1

and

key2: value2

with your actual node selector key-value pairs. ### Steps to Apply the Configuration: 1. Update the

values.yaml

File: Add the

global.jobs.kube.nodeSelector

configuration to your

values.yaml

file as shown in the example above. 2. Apply the Helm Chart: Use Helm to apply the updated chart to your EKS cluster. For example:

sh     helm upgrade --install airbyte ./airbyte -f values.yaml

3. Verify the Configuration: Ensure that the job pods are running on the nodes that match the specified node selectors by checking the node architecture of the running pods. ### Summary of Relevant Information: -

global.jobs.kube.nodeSelector

: This configuration allows you to define node selectors for job pods directly in the Helm chart values file. Each key-value pair specifies a node selector criterion. By setting the

global.jobs.kube.nodeSelector

configuration, you can ensure that the job pods launched by the worker run on nodes that match the specified criteria, such as ARM architecture. If you encounter further issues, you may need to check the specific images being used and ensure they are compatible with the target architecture. You can also refer to the official Airbyte documentation for more detailed guidance on configuring node selectors and handling mixed architecture clusters.