Using these two variables didn't work, my SFTP connector always crashes my sync with a How to solve this problem a Java.lang.OutOfMemoryError: Java heap space. How to solve this bug? JOB_MAIN_CONTAINER_MEMORY_REQUEST JOB_MAIN_CONTAINER_MEMORY_LIMIT

To address the

Java.lang.OutOfMemoryError: Java heap space

error when using the SFTP connector in Airbyte, you can follow these steps to properly configure the memory settings: 1. Set Environment Variables: Ensure that you have correctly set the

JOB_MAIN_CONTAINER_MEMORY_REQUEST

and

JOB_MAIN_CONTAINER_MEMORY_LIMIT

environment variables in your

.env

file. The units should be specified correctly as per Docker's documentation. For example:

plaintext     JOB_MAIN_CONTAINER_MEMORY_REQUEST=4g     JOB_MAIN_CONTAINER_MEMORY_LIMIT=6g

2. Restart Airbyte: After updating the

.env

file, restart your Airbyte instance to apply the changes. 3. Check Sync Logs: Verify in the sync logs that the memory settings are being applied. You should see log entries indicating the memory request and limit settings for the Docker container. 4. Java Connector Memory Behavior: Be aware that Java connectors in Airbyte follow Java’s default behavior with container memory and will only use up to 1/4 of the host’s allocated memory. This means that if your host has 8GB of RAM, a Java connector will limit itself to 2GB of RAM. You may need to adjust the host's allocated memory or the environment variables accordingly. 5. Manual Garbage Collection: As a temporary workaround, you can manually trigger the Java Garbage Collector (GC) in the worker container to free up memory. Here are the steps:

bash     docker exec -it airbyte-worker /bin/bash     jcmd 1 GC.run

6. Adjust JVM Options: Another workaround is to update the default JVM options to manage non-heap memory consumption in Java. You can set the following options:

plaintext     DEFAULT_JVM_OPTS='-XX:+ExitOnOutOfMemoryError -XX:MaxRAMPercentage=75.0'

7. Monitor and Adjust: Continuously monitor the memory usage and adjust the settings as needed. If the problem persists, consider following the related GitHub issues for updates and potential fixes. For more detailed information, you can refer to the following sources: - [Scaling Airbyte Documentation](https://docs.airbyte.com/operator-guides/scaling-airbyte#memory) - [Forum Post on Memory Parameters](https://discuss.airbyte.io/t/1399) - [GitHub Issue on Memory Bug](https://github.com/airbytehq/airbyte/issues/31694) By following these steps, you should be able to mitigate the

Java.lang.OutOfMemoryError

and ensure smoother sync operations with your SFTP connector.