@kapa.ai Can we change the default JVM behavior for source connector in Java to only use 1/4 of the available memory wihtin the container

Yes, you can change the default JVM behavior for source connectors in Java to use more than the default 1/4 of the available memory within the container. By default, Java connectors in Airbyte will only use up to 1/4 of the host's allocated memory. This can be customized by setting the

JOB_MAIN_CONTAINER_MEMORY_REQUEST

and

JOB_MAIN_CONTAINER_MEMORY_LIMIT

environment variables to custom values. Here is how you can do it: 1. Set Environment Variables: You can set the

JOB_MAIN_CONTAINER_MEMORY_REQUEST

and

JOB_MAIN_CONTAINER_MEMORY_LIMIT

environment variables to specify the memory allocation for the Java connectors. For example, if you want to allocate more memory, you can set these environment variables in your Docker or Kubernetes configuration:

yaml # Example for Docker environment:   JOB_MAIN_CONTAINER_MEMORY_REQUEST: "4g"   JOB_MAIN_CONTAINER_MEMORY_LIMIT: "4g"

yaml # Example for Kubernetes env: - name: JOB_MAIN_CONTAINER_MEMORY_REQUEST   value: "4g" - name: JOB_MAIN_CONTAINER_MEMORY_LIMIT   value: "4g"

2. Update Docker Compose or Kubernetes Configuration: Ensure that these environment variables are included in your Docker Compose or Kubernetes configuration files. For more detailed information, you can refer to the [Airbyte documentation on scaling](https://docs.airbyte.com/operator-guides/scaling-airbyte#memory). ### Relevant Information from Knowledge Sources - Scaling Airbyte Documentation: "You may want to customize this by setting

JOB_MAIN_CONTAINER_MEMORY_REQUEST

and

JOB_MAIN_CONTAINER_MEMORY_LIMIT

environment variables to custom values." [Source](https://docs.airbyte.com/operator-guides/scaling-airbyte#memory) - Forum Post: "You can set

JOB_MAIN_CONTAINER_MEMORY_REQUEST=28g

." [Source](https://discuss.airbyte.io/t/1399) By following these steps, you can adjust the memory allocation for your Java source connectors to better suit your needs and avoid out-of-memory exceptions.