Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Where do I set JOB_MAIN_CONTAINER_CPU_REQUEST when using abctl and a values.yaml file?

You can set the `JOB_MAIN_CONTAINER_CPU_REQUEST` in the `values.yaml` file when deploying Airbyte on Kubernetes using Helm. This environment variable defines the job container's minimum CPU usage.   Here is an example of how you can set it:  ```yaml global:   jobs:     resources:       requests:         cpu: 250m  # This line sets JOB_MAIN_CONTAINER_CPU_REQUEST ```  After setting the values in the `values.yaml` file, you need to uninstall and reinstall Airbyte for the changes to take effect. This is because changing the configmap doesn't cause Kubernetes to re-create the pods with updated environment variable values.   If you're using `abctl`, you might encounter an error if it's unable to read values from the `values.yaml` file. In that case, verify that you are in the correct directory or informing the path to the file correctly.  Sources: - [Github Issue (open): [helm] Excessive cpu and memory limits in job containers &gt; Comments Part 2](<https://github.com/airbytehq/airbyte/issues/35749>) - [Documentation &gt; Deploying-airbyte &gt; Troubleshooting abctl &gt; Common Errors &gt; Unable to Read Values from YAML file](<https://docs.airbyte.com/deploying-airbyte/troubleshoot-deploy#unable-to-read-values-from-yaml-file>)

And how do I set NORMALIZATION_JOB_MAIN_CONTAINER_MEMORY_LIMIT? <@U04SE163WC8>

The `NORMALIZATION_JOB_MAIN_CONTAINER_MEMORY_LIMIT` is an environment variable that you can set in your Airbyte deployment to define the maximum RAM usage for the normalization job container. The units for this setting follow either Docker or Kubernetes, depending on your deployment.  You can set this variable in your `.env` file or directly in your deployment configuration. The exact method can vary depending on your specific deployment method (Docker, Kubernetes, etc.).   Here is an example of how you might set it in a `.env` file:  ```bash NORMALIZATION_JOB_MAIN_CONTAINER_MEMORY_LIMIT=2g ```  In this example, the normalization job container's maximum RAM usage is set to 2 gigabytes.  Please note that the actual value you should use depends on the resources available in your environment and the requirements of your data workloads.  Sources: - [Configuring Connector Resources](<https://docs.airbyte.com/operator-guides/configuring-connector-resources#configuring-instance-wide-requirements>) - [Github Issue 30814](<https://github.com/airbytehq/airbyte/issues/30814>) - [Github Issue 33693](<https://github.com/airbytehq/airbyte/issues/33693>)

Normalization is deprecated. It isn’t used in `abctl`.