Aviad Deri
12/25/2025, 11:56 AMsecretName where password is stored
passwordSecretKey: "password" # e.g."database-password"
workloads:
containerOrchestrator:
secretName: "airbyte-config-secrets"
secretMountPath: "/secrets/gcs-log-creds"
auth:
enabled: true
instanceAdmin:
firstName: "devops"
lastName: "team"
email: "devops@domain.com"
storage:
type: "gcs"
secretName: "airbyte-config-secrets"
bucket: # GCS bucket names that you've created. We recommend storing the following all in one bucket.
log: ${bucket_name}
state: ${bucket_name}
workloadOutput: ${bucket_name}
gcs:
projectId: ${project_id}
credentialsJsonPath: /secrets/gcs-log-creds/gcp.json
jobs:
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 1Gi
serviceAccount:
# -- Specifies whether a ServiceAccount should be created
create: true
# -- Annotations for service account. Evaluated as a template. Only used if create is true.
annotations:
"iam.gke.io/gcp-service-account": "${gcp_service_account}"
# -- Name of the service account to use. If not set and create is true, a name is generated using the fullname template.
name: "${k8s_service_account_name}"
postgresql:
enabled: false
temporal:
extraEnv:
- name: SKIP_DB_CREATE
value: "true"
- name: DBNAME
value: temporal_qa
- name: VISIBILITY_DBNAME
value: temporal_visibility_qa
- name: POSTGRES_TLS_ENABLED
value: "false"
- name: SQL_TLS_ENABLED
value: "false"
still in logs i see
2025-12-25 134823 platform INFO [source] image: airbyte/source-postgres:3.7.0 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=1Gi, cpu=1}, additionalProperties={})
2025-12-25 134823 platform INFO [destination] image: airbyte/destination-bigquery:3.0.16 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=1Gi, cpu=1}, additionalProperties={})
2025-12-25 134823 platform INFO [orchestrator] image: airbyte/container-orchestrator:2.0.1 resources: ResourceRequirements(claims=[], limits={memory=2Gi, cpu=2}, requests={memory=2Gi, cpu=2}, additionalProperties={})
why?kapa.ai
12/25/2025, 11:56 AMglobal.jobs.resources configuration is being ignored in version 2.0.1, but they do highlight several known behaviors and issues that explain why your replication pods are defaulting to higher values. Based on the provided documents, here are the likely reasons your configuration is not being applied: ### 1. Connector-Specific Overrides In recent versions of Airbyte, resource requirements defined at the actor (connector) level or connection level take precedence over global settings. Even if you set global.jobs.resources, these values may be overridden by: * Actor Definitions: Specific connectors (like source-postgres or destination-bigquery) often have hardcoded default resource requirements in the Airbyte database [[workload resource requests and limits not applied](https://github.com/airbytehq/airbyte/issues/68162)]. * Connection-level settings: Resource requirements can be set specifically for a single connection, which overrides global defaults [[Orchestrator container gets same resource limit/request values](https://discuss.airbyte.io/t/3775)]. ### 2. Known Parsing Issues with Millicore Values There is a documented issue in Airbyte (reported for version 2.0.1) where the workload-launcher fails to correctly parse millicore values (e.g., 500m, 1000m) and falls back to default values (typically cpu: 2). * Workaround: Try setting your CPU values as integers (e.g., cpu: "1") instead of millicores (1000m) in your Helm values [[workload resource requests and limits not applied](https://github.com/airbytehq/airbyte/issues/68162)]. ### 3. Use of workloads.resources in newer versions For Airbyte versions 2.x, some documentation and issues suggest using the workloads.resources block rather than global.jobs.resources to manage replication settings. You may need to disable connector defaults explicitly: yaml workloads: resources: useConnectorResourceDefaults: false replication: memory: request: "1Gi" limit: "1Gi" cpu: request: "1" # Use integer instead of "500m" due to parsing bugs limit: "1" [[workload resource requests and limits not applied](https://github.com/airbytehq/airbyte/issues/68162)] ### How to Verify To confirm which configuration is being detected, you can continue checking the job logs as you have been doing. Airbyte logs the specific ResourceRequirements for the source, destination, and orchestrator as they are launched [[Debugging Connection Resources](https://docs.airbyte.com/platform/2.0/operator-guides/configuring-connector-resources#debugging)]. If the logs still show cpu=2 after changing to integer values, it highly suggests the connector-specific defaults in the database are overriding your global Helm values.Aviad Deri
12/26/2025, 9:45 AMkapa.ai
12/26/2025, 9:45 AMkapa.ai
12/26/2025, 9:45 AMworkloads block rather than the jobs block. Based on the provided documentation and GitHub issues, here is how you can force the limits for replication pods: ### 1. Use the workloads block and Disable Defaults In Airbyte 2.x, the global.jobs.resources configuration is often ignored in favor of global.workloads.resources. You must also explicitly set useConnectorResourceDefaults: false to prevent the system from using the hardcoded defaults (like CPU: 2) found in the database [[workload resource requests and limits not applied](https://github.com/airbytehq/airbyte/issues/68162)]. Try updating your values.yaml to this structure: yaml global: workloads: resources: useConnectorResourceDefaults: false replication: memory: request: "1Gi" limit: "1Gi" cpu: # Use integers ("1") instead of millicores ("1000m") to avoid known parsing bugs request: "1" limit: "1" ### 2. Avoid Millicore Values ("m") The knowledge sources highlight a specific bug where the workload-launcher fails to parse values like 500m or 1000m, causing it to fall back to the default of cpu: 2. Using whole integers (e.g., "1") for CPU settings is the recommended workaround [[workload resource requests and limits not applied](https://github.com/airbytehq/airbyte/issues/68162)]. ### 3. Update Connector Definitions via SQL If the Helm changes still do not reflect in the logs, it is because the "Connector Definition" requirements in your database are taking precedence [[Configuring Connector Resources](https://docs.airbyte.com/platform/2.0/operator-guides/configuring-connector-resources)]. You can override these by running SQL against your external PostgreSQL database: 1. Find the definition ID for your connector: sql SELECT id, name FROM actor_definition WHERE docker_repository LIKE '%source-postgres%'; 2. Update the resource requirements directly in the database: sql UPDATE actor_definition SET resource_requirements = '{"jobSpecific": [{"jobType": "sync", "resourceRequirements": {"cpu_limit": "1", "cpu_request": "0.5", "memory_limit": "1Gi", "memory_request": "1Gi"}}]}' WHERE id = '<ID_FROM_STEP_1>'; [[Configuring Connector Resources](https://docs.airbyte.com/platform/2.0/operator-guides/configuring-connector-resources)] ### 4. GKE Autopilot Considerations Note that GKE Autopilot has specific minimums and ratios for CPU and Memory. If your requested values are too low or don't meet Autopilot's resource ratios, GKE may automatically scale them up, which might explain why the pods still show higher values than your configuration.