Aman Raj
10/25/2023, 6:53 PMkapa.ai
10/25/2023, 6:54 PMJOB_MAIN_CONTAINER_MEMORY_REQUEST and JOB_MAIN_CONTAINER_MEMORY_LIMIT environment variables are used to define the minimum and maximum RAM usage for the job container in Airbyte [[source](https://docs.airbyte.com/operator-guides/configuring-connector-resources)]. However, it's important to note that Java connectors, which include all Source database connectors, follow Java's default behavior with container memory and will only use up to 1/4 of the host's allocated memory [[source](https://docs.airbyte.com/operator-guides/scaling-airbyte)]. For example, on a Docker agent with 8GBs of RAM configured, a Java connector limits itself to 2GBs of RAM and will see Out-of-Memory exceptions if this goes higher. This could be why you're seeing a 2GB limit in Docker stats, even though you've set the JOB_MAIN_CONTAINER_MEMORY_LIMIT to 500MB. If you want to customize this, you can set the JOB_MAIN_CONTAINER_MEMORY_REQUEST and JOB_MAIN_CONTAINER_MEMORY_LIMIT environment variables to custom values. However, be aware that users currently need to over-specify memory resource for Java connectors [[source](https://docs.airbyte.com/operator-guides/scaling-airbyte)]. If you're still facing issues, you can check the job logs in the Airbyte UI to confirm if the right configuration is being detected [[source](https://docs.airbyte.com/operator-guides/configuring-connector-resources)]. If a job is running out-of-memory, the logs will show the resource requirements as containers are created.