Hi, I need help with Requests and Limits for `dest...
# replication-ask-ai
h
Hi, I need help with Requests and Limits for
destination
and
source
pods. I deploy Airbyte using Helm latest charts, and I see that my
destination
and
source
pods for some connections are requesting too little for their loads, even passing the limits. In the Helm charts, there are no specific config for
destination
and
source
. It seems that the equivalent to tweak these pods is the
jobs
resources. Yet, even when I change the request for
jobs
, I don't see these requests being reflected on destination and source pods.
k
A message from kapa.ai
To set resource limits for connector pods in Airbyte deployed on Kubernetes, you need to configure the Docker resource limits section of the
.env
file in the
kube/overlays
directory. You can find more information about setting resource limits on connector pods in the [Airbyte documentation](https://docs.airbyte.com/deploying-airbyte/on-kubernetes/#setting-resource-limits). Additionally, you can modify the
.env
variables
JOB_MAIN_CONTAINER_MEMORY_REQUEST
and
JOB_MAIN_CONTAINER_MEMORY_LIMIT
to adjust the resources for the main container. Here's an example of how a user set them in a [forum discussion](https://discuss.airbyte.io/t/increase-sync-worker-resources/1714/3). To check if the right configuration is being detected, you can navigate to the Job in the Airbyte UI and look for the log. On Kubernetes, the log will look something like this:
Copy code
2022-08-12 01:22:20 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = source-intercom-check-480195-0-abvnr for airbyte/source-intercom:0.1.24 with resources io.airbyte.config.ResourceRequirements@11cc9fb9[cpuRequest=2,cpuLimit=2,memoryRequest=200Mi,memoryLimit=200Mi]
For more information on customizing your deployment using Helm charts, you can refer to the [Airbyte documentation on deploying on Kubernetes via Helm](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm). Create a
values.yaml
file in your local folder and populate it with the default configuration override values. After specifying your own configuration, run the following command:
Copy code
helm install --values path/to/values.yaml %release_name% airbyte/airbyte
Please note that the information provided is based on the available extracts, and there might be some differences in the actual implementation.