@kapa.ai I have opensource airbyte deployed on an EC2 instance (type t3.xlarge with 4 CPUs and 16 GiB in mem). When I first installed abctl and tried running a replication instance for CDC (postgres to snowflake) I got the following error ERROR Stage Pipeline Exception: io.airbyte.workload.launcher.pipeline.stages.model.StageError: io.airbyte.workers.exception.ResourceConstraintException: Unable to start the REPLICATION pod. This may be due to insufficient system resources. Please check available resources and try again. message: io.airbyte.workers.exception.ResourceConstraintException: Unable to start the REPLICATION pod. This may be due to insufficient system resources. Please check available resources and try again. stackTrace: [Ljava.lang.StackTraceElement;@7447b01e I then re-installed abctl on low resource mode and that worked, but the sync times after the initial loads on my connections have been quite slow. So my question are 1) Would running this not on low resource mode improve those sync times? 2) What size EC2 instance is recommended to allow the replication pod to have enough resources to start?

Addressing Your ResourceConstraintException and Sync Performance 1) Would running without low-resource mode improve sync times? Potentially yes. The

--low-resource-mode

flag restricts the resources available to connector pods, which can directly impact sync throughput. More CPU and memory available to the replication pods generally correlates with faster syncs. [Scaling Airbyte] Airbyte's documentation also notes that CPU performance is directly correlated with sync speed — higher-performance CPUs lead to better outcomes. [Faster sync speed] 2) Recommended EC2 instance size The knowledge sources don't give a specific recommendation for

abctl

-based deployments with CDC workloads. However: • A t3.xlarge (4 vCPU / 16 GiB) should in principle be sufficient, but the

ResourceConstraintException

you saw suggests the replication pods' default resource requests were exceeding what was schedulable in your cluster — likely because Airbyte's own platform pods were already consuming a significant portion of the available resources, leaving insufficient headroom for the replication pod. • The default replication pod resources are

limits={memory=2Gi, cpu=2}, requests={memory=1Gi, cpu=1}

per container (source + destination + orchestrator), which can add up quickly on a single node. A t3.2xlarge (8 vCPU / 32 GiB) or larger would give you more headroom. The general recommendation from Airbyte is to start with a mid-sized instance (4–8 cores) and tune from there. [Scaling Airbyte] Additional options to try on your current instance Rather than immediately upsizing, you could try reducing the default resource requests for connector pods in your

values.yaml

so they fit within your available capacity:

global:
  jobs:
    resources:
      limits:
        cpu: "1"
        memory: 2Gi
      requests:
        cpu: "500m"
        memory: 1Gi

This is especially useful if your requested CPU/memory is blocking scheduling while actual usage is low. [Enterprise Scaling Guide] > ⚠️ Note: There is an open GitHub issue reporting that resource configurations set via

values.yaml

may be ignored by

abctl

v2.x, with the workload-launcher defaulting to hardcoded values (2Gi/2CPU). As a workaround, you can try setting env vars directly on the workload-launcher: [Resource limits bug] >

workloadLauncher:
>   extraEnv:
>     - name: JOB_MAIN_CONTAINER_CPU_LIMIT
>       value: "1"
>     - name: JOB_MAIN_CONTAINER_CPU_REQUEST
>       value: "500m"
>     - name: JOB_MAIN_CONTAINER_MEMORY_LIMIT
>       value: "2Gi"
>     - name: JOB_MAIN_CONTAINER_MEMORY_REQUEST
>       value: "1Gi"

In summary, upsizing to an 8-core / 32 GiB instance is the most straightforward path to running without low-resource mode and improving sync performance for CDC workloads. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.