Hi - I am having some issues with Airbyte on Kubernetes Kubernetes. It seems that the source and connection pods that are spun up occasionally get killed. I think that's because of the memory allocation reserved for them (which is likely too low). However, I am not entirely sure how to configure that? • if I look at the Helm chart, I see parameters for the

worker

deployment - however I think that is not the actual worker syncing the data? but rather the service that spins up the actual workers? • Then I stumbled upon this page - which seems more like what I need. Does that I mean I need to set those env variables in the

worker

env vars config in the helm chart? Also, the env vars are only about cpu and mem resources, how can I add taunts and tolerations too? EDIT: As a side note, I feel those resource configs should be exposed in the helm chart values my bad it may actually already be in the helm chart, under the keys of

jobs

?

You're correct in your understanding that the

worker

deployment is responsible for spinning up the actual workers that perform the data syncing. The resource allocation for these workers can indeed be configured via environment variables, as mentioned in the [Configuring Connector Resources](https://docs.airbyte.com/operator-guides/configuring-connector-resources) page. The environment variables you need to set are: -

JOB_MAIN_CONTAINER_CPU_REQUEST

- Define the job container's minimum CPU usage. -

JOB_MAIN_CONTAINER_CPU_LIMIT

- Define the job container's maximum CPU usage. -

JOB_MAIN_CONTAINER_MEMORY_REQUEST

- Define the job container's minimum RAM usage. -

JOB_MAIN_CONTAINER_MEMORY_LIMIT

- Define the job container's maximum RAM usage. These variables can be set in the

.env

file as per the [Configuring Airbyte](https://docs.airbyte.com/operator-guides/configuring-airbyte) guide. If you're using the Helm chart for deployment, you would need to include these environment variables in the

values.yaml

file as per the [Deploy Airbyte on Kubernetes using Helm](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm) guide. As for adding taints and tolerations, this is not directly covered in the provided extracts. However, in general, taints and tolerations in Kubernetes are usually configured at the pod level in the pod specification. You might need to modify the Helm chart or the Kubernetes manifests directly to include these configurations. Lastly, your feedback about exposing resource configs in the Helm chart values is valuable. It would indeed make the configuration process more straightforward. You might want to consider raising this as a feature request in the Airbyte GitHub repository or discuss it in the Airbyte community forum.