https://linen.dev logo
Join Slack
Powered by
```[initContainer] image: airbyte/workload-init-co...
# replication-ask-ai
r
Copy code
[initContainer] image: airbyte/workload-init-container:1.7.2 resources: ResourceRequirements(claims=[], limits={}, requests={memory=0, cpu=0}, additionalProperties={})
k
The log line you're seeing:
Copy code
[initContainer] image: airbyte/workload-init-container:1.7.2 resources: ResourceRequirements(claims=[], limits={}, requests={memory=0, cpu=0}, additionalProperties={})
indicates that the 
workload-init-container
is launching with no resource limits and zero resource requests. This is a known pattern in Airbyte where resource configurations may not be properly propagated to init containers or job pods. What this means • 
limits={}
— no CPU/memory limits are set • 
requests={memory=0, cpu=0}
— zero values are explicitly set for requests This can cause issues if your cluster has an OPA (Open Policy Agent) or admission webhook policy that requires resource limits/requests to be defined on all containers. [OPA resource issue] How to configure resource requirements You can set instance-wide resource requirements using the following environment variables in your `values.yaml`: [Configuring Connector Resources]
Copy code
env_vars:
  JOB_MAIN_CONTAINER_CPU_REQUEST: "0.5"
  JOB_MAIN_CONTAINER_CPU_LIMIT: "1"
  JOB_MAIN_CONTAINER_MEMORY_REQUEST: "512Mi"
  JOB_MAIN_CONTAINER_MEMORY_LIMIT: "1Gi"
Or alternatively using the 
global.workloads.resources
structure:
Copy code
global:
  workloads:
    resources:
      replication:
        cpu:
          request: "2"
          limit: "4"
        memory:
          request: 4Gi
          limit: 8Gi
Debugging tip To confirm the correct configuration is being picked up, check the job logs for the 
[source]
, 
[destination]
, and 
[orchestrator]
container lines — they should show non-zero resource values if your configuration is applied correctly. [Debugging Resources] Known issue There is an open GitHub issue reporting that resource limits set via 
values.yaml
, connection overrides, and env vars may be ignored in some 
abctl
-based deployments, with pods defaulting to hardcoded values. [Resource limits ignored] If you're seeing 
0
values despite configuration, this may be a related bug worth tracking. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai same job I got this 10 seconds later
Copy code
2026-05-26 16:59:12 platform INFO [source] image: airbyte/source-mysql:3.1.1 resources: ResourceRequirements(claims=[], limits={memory=4Gi, cpu=3}, requests={memory=0, cpu=0}, additionalProperties={})
2026-05-26 16:59:12 platform INFO [destination] image: airbyte/destination-snowflake:3.15.10 resources: ResourceRequirements(claims=[], limits={memory=4Gi, cpu=3}, requests={memory=0, cpu=0}, additionalProperties={})
2026-05-26 16:59:12 platform INFO [orchestrator] image: airbyte/container-orchestrator:1.7.2 resources: ResourceRequirements(claims=[], limits={memory=4Gi, cpu=3}, requests={memory=0, cpu=0}, additionalProperties={})
k
Your logs show that all three containers (source, destination, orchestrator) have: • 
limits={memory=4Gi, cpu=3}
— limits are being applied correctly • 
requests={memory=0, cpu=0}
— requests are explicitly set to zero What's happening The limits are working as expected. The 
requests=0
pattern means no meaningful resource requests are being set, which tells the Kubernetes scheduler that these pods require essentially no resources to be scheduled. This is not necessarily a functional problem for running the sync, but it does have implications: • The Kubernetes scheduler won't reserve resources for these pods, which can lead to resource contention on busy nodes • Some clusters with strict admission policies may reject pods with zero requests • It can cause OOM kills if the node is overcommitted, since Kubernetes won't account for the pod's actual memory usage when scheduling other workloads What you should configure To set non-zero requests, you need to explicitly configure them. Based on the knowledge sources, you can do this via `values.yaml`: [Configuring Connector Resources]
Copy code
global:
  env_vars:
    JOB_MAIN_CONTAINER_CPU_REQUEST: "1"
    JOB_MAIN_CONTAINER_CPU_LIMIT: "3"
    JOB_MAIN_CONTAINER_MEMORY_REQUEST: "2Gi"
    JOB_MAIN_CONTAINER_MEMORY_LIMIT: "4Gi"
Or alternatively, set requests directly at the connection level via SQL: [workload resource limits]
Copy code
UPDATE connection
SET resource_requirements = jsonb_build_object(
  'cpu_request', '1',
  'cpu_limit',   '3',
  'memory_request', '2Gi',
  'memory_limit',  '4Gi'
)
WHERE id = '<your-connection-id>';
Important caveat There is an open bug where resource requests set via 
values.yaml
may be ignored by the workload-launcher, even when limits are correctly applied. [Resource limits ignored] [orchestrator memory ignored] The fact that your limits are showing correctly but requests remain at 
0
is consistent with this known issue. The database-level SQL workaround has been confirmed to work by multiple users in these cases. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai at this another instance the resource request is too high, I cannot run two jobs at the same time because of that...can I set this request resources such as the prior instance:
Copy code
2026-05-27 15:10:45 platform INFO [initContainer] image: airbyte/workload-init-container:2.1.0 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=2Gi, cpu=2}, additionalProperties={})
2026-05-27 15:10:45 platform INFO Launching replication pod: replication-job-18900-attempt-0 (selectors = {}) with containers:
2026-05-27 15:10:45 platform INFO [source] image: airbyte/source-mysql:3.1.1 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=1Gi, cpu=1}, additionalProperties={})
2026-05-27 15:10:45 platform INFO [destination] image: airbyte/destination-snowflake:3.15.10 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=2Gi, cpu=1}, additionalProperties={})
2026-05-27 15:10:45 platform INFO [orchestrator] image: airbyte/container-orchestrator:2.1.0 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=2Gi, cpu=2}, additionalProperties={})
k
On this instance, the resource requests are high (especially the orchestrator at 
requests={memory=2Gi, cpu=2}
), which is preventing two jobs from running concurrently. You want to reduce these requests to match the lower values seen on your other instance (
requests={memory=0, cpu=0}
). Recommended approach 1. Via values.yaml (instance-wide) Try setting requests explicitly to lower values: [Configuring Connector Resources]
Copy code
global:
  env_vars:
    JOB_MAIN_CONTAINER_CPU_REQUEST: "0.5"
    JOB_MAIN_CONTAINER_CPU_LIMIT: "2"
    JOB_MAIN_CONTAINER_MEMORY_REQUEST: "512Mi"
    JOB_MAIN_CONTAINER_MEMORY_LIMIT: "2Gi"
Or using the v2 chart structure:
Copy code
global:
  workloads:
    resources:
      useConnectorResourceDefaults: false
      replication:
        cpu:
          request: "0.5"
          limit: "2"
        memory:
          request: "512Mi"
          limit: "2Gi"
2. Via SQL at the connection level (most reliable workaround) Multiple users have confirmed this works even when 
values.yaml
settings are ignored: [workload resource limits]
Copy code
UPDATE connection
SET resource_requirements = jsonb_build_object(
  'cpu_request',    '0.5',
  'cpu_limit',      '2',
  'memory_request', '512Mi',
  'memory_limit',   '2Gi'
)
WHERE id = '<your-connection-id>';
Important caveats • There is a known open bug where resource configurations set via 
values.yaml
and env vars are ignored by the workload-launcher, which defaults to hardcoded values (
2Gi/2CPU
). [Resource limits ignored] • The SQL workaround applies to source and destination containers, but the orchestrator container (
requests={memory=2Gi, cpu=2}
in your logs) currently has no reliable override path — this is a separate open bug. [orchestrator memory ignored] • One community workaround for the orchestrator specifically is a Kubernetes Mutating Webhook that patches the orchestrator container resources at pod creation time. [orchestrator memory ignored] The orchestrator's high requests (
cpu=2, memory=2Gi
) are likely the main blocker for running two concurrent jobs, and unfortunately that is the hardest to override given the current bugs. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
r
@kapa.ai why don't you set as 0?
k
Good question. Setting requests to 
0
(as seen on your other instance) is technically possible, but it comes with trade-offs that make it a poor recommendation: • Resource contention risk: With 
requests=0
, the Kubernetes scheduler doesn't reserve any resources for the pod. If the node becomes overcommitted, your sync pods are the first candidates for OOM kills since Kubernetes has no accounting of their expected usage. • QoS class degradation: Pods with 
requests=0
and non-zero limits are placed in the Burstable QoS class, which offers weaker guarantees than Guaranteed (where requests = limits). This makes them more likely to be evicted under node pressure. That said, the knowledge sources do not explicitly document why 
requests=0
appears in one instance but not the other — this appears to be a side effect of the known bug where resource configurations are inconsistently applied by the workload-launcher. [Resource limits ignored] The recommended approach is to set low but non-zero requests (e.g., 
cpu=0.5
, 
memory=512Mi
) so that: 1. The scheduler can make informed placement decisions 2. You can fit multiple jobs on the same node 3. Pods still have headroom to burst up to their limits If the 
values.yaml
approach is being ignored due to the known bug, the SQL workaround at the connection level is currently the most reliable way to achieve this. [workload resource limits] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
2 Views
Open in Slack
PreviousNext
Powered by