< kapa ai> for an Airbyte application running on a Kubernete Airbyte #replication-ask-ai

<@U04S8PLJWMT> for an Airbyte application running ...

Jase Pramod

04/02/2026, 8:21 PM

@kapa.ai for an Airbyte application running on a Kubernetes setup. I want to create a separate node group just for sync pods with spot instances to reduce cost on infrastructure. Does it work?

kapa.ai

04/02/2026, 8:24 PM

Yes, this is a supported and recommended pattern in Airbyte. You can configure a dedicated node group for job/sync-related pods (connector pods) while keeping static workloads (like

server

webapp

temporal

, etc.) on a separate node group. This is explicitly recommended to reduce the blast radius of an underprovisioned deployment and ensure UI/API availability is not impacted by concurrent syncs. How to Configure It You need to use node selectors to direct connector pods to your spot instance node group. Here's an example using Helm chart V1 `values.yaml`:

Copy code

server:
  nodeSelector:
    type: static

temporal:
  nodeSelector:
    type: static

webapp:
  nodeSelector:
    type: static

worker:
  nodeSelector:
    type: jobs

workload-launcher:
  nodeSelector:
    type: static
  ## Pods spun up by the workload launcher will run in the 'jobs' node group.
  extraEnv:
    - name: JOB_KUBE_NODE_SELECTORS
      value: type=jobs
    - name: SPEC_JOB_KUBE_NODE_SELECTORS
      value: type=jobs
    - name: CHECK_JOB_KUBE_NODE_SELECTORS
      value: type=jobs
    - name: DISCOVER_JOB_KUBE_NODE_SELECTORS
      value: type=jobs

orchestrator:
  nodeSelector:
    type: jobs

workload-api-server:
  nodeSelector:
    type: jobs

Or using Helm chart V2:

Copy code

global:
  jobs:
    kube:
      nodeSelector:
        type: jobs
      scheduling:
        check:
          nodeSelectors:
            type: jobs
        discover:
          nodeSelectors:
            type: jobs
        spec:
          nodeSelectors:
            type: jobs

server:
  nodeSelector:
    type: static

temporal:
  nodeSelector:
    type: static

workloadLauncher:
  nodeSelector:
    type: static

worker:
  nodeSelector:
    type: jobs

workloadApiServer:
  nodeSelector:
    type: jobs

[Multiple Node Groups] Key Points • The workload-launcher
is the component responsible for spinning up connector pods. Setting

JOB_KUBE_NODE_SELECTORS

(and the job-specific variants) on it ensures that sync pods land on your designated node group (i.e., your spot instances). [Multiple Node Groups] • Make sure to also add tolerations if your spot instance node group has taints, so the pods can be scheduled there. [Node Selectors on Helm Chart] • Note that

SPEC_JOB_KUBE_NODE_SELECTORS

CHECK_JOB_KUBE_NODE_SELECTORS

, and

DISCOVER_JOB_KUBE_NODE_SELECTORS

must be set explicitly — the fallback from

JOB_KUBE_NODE_SELECTORS

may not work for non-sync jobs. [Node Selectors on Helm Chart] In summary, this approach works well and is a recommended cost-optimization strategy for Airbyte on Kubernetes. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

3 Views

Open in Slack

Previous Next