Slackbot
10/23/2024, 6:16 PMuser
10/23/2024, 6:16 PMabctl
, you can try running Airbyte in low resource mode. This mode is designed to work with 2 CPUs and 8 GB of RAM, which your instance exceeds. You can use the following command: abctl local install --low-resource-mode
[Source](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart#suggested-resources) 2. Check Resource Allocation: Ensure that Docker is allowed to use the full resources of your EC2 instance. By default, Docker doesn't limit resource usage among containers, but it's worth verifying your Docker settings. 3. Increase Worker Limits: You can try increasing the number of workers by setting the following environment variables: AIRBYTE__MAX_DISCOVER_WORKERS=10 AIRBYTE__MAX_SYNC_WORKERS=10
[Source](https://github.com/airbytehq/airbyte/issues/44833) 4. Adjust Job Container Resources: You can fine-tune the resources allocated to job containers by setting these environment variables: JOB_MAIN_CONTAINER_MEMORY_REQUEST JOB_MAIN_CONTAINER_MEMORY_LIMIT JOB_MAIN_CONTAINER_CPU_REQUEST JOB_MAIN_CONTAINER_CPU_LIMIT
[Source](https://discuss.airbyte.io/t/861) 5. Check System Load: Ensure that no other processes on your EC2 instance are consuming significant resources that might prevent Airbyte from starting its sync process. 6. Verify Docker Volume: Make sure you have sufficient space in your Docker volumes and that there are no issues with Docker's storage driver. If these steps don't resolve the issue, you may want to check the Airbyte server logs for more detailed error messages. You can access these logs using: docker logs airbyte-server
If the problem persists, consider opening a new issue on the [Airbyte GitHub repository](https://github.com/airbytehq/airbyte) with detailed logs and your configuration for further assistance.user
10/23/2024, 6:20 PMJOB_MAIN_CONTAINER_MEMORY_REQUEST=<value> JOB_MAIN_CONTAINER_MEMORY_LIMIT=<value> JOB_MAIN_CONTAINER_CPU_REQUEST=<value> JOB_MAIN_CONTAINER_CPU_LIMIT=<value>
For memory, you can use units like "2g" for 2 gigabytes. For CPU, you express it as a percentage of a single core, e.g., "0.50" for 50% of a core [Forum Post: How to specify CPU,MEMORY limit in .env](https://discuss.airbyte.io/t/1675). If you're using a deployment method that uses a .env file or similar configuration, you would typically modify these values there. However, for the specific method of changing these using abctl in version 1.1.0, I don't have accurate information to provide.