@kapa.ai i want to sync around 4K rows from sql server to snowflake & Airbyte 1.1.0 is running on ec2 instance with 8 CPUs, 32 GB Memory & pipeline is kept on running for hours with this in logs:

Workload dbf246d7-8seg-4c20-b226-2d0fec5f6454_135_0_sync is pending

Based on the information provided in the knowledge sources, I can offer some insights into your situation with Airbyte syncing from SQL Server to Snowflake. It appears that you're experiencing a common issue where syncs can get stuck or take an unusually long time to complete, even with relatively small amounts of data. Several users have reported similar problems with Airbyte syncs taking too long or getting stuck. For example: 1. In [this forum post](https://discuss.airbyte.io/t/995), a user reported that syncing 300 tables with less than 2MB of data took 3 hours, even with 4 vCPU and 32 GB RAM. 2. Another user in [this GitHub issue](https://github.com/airbytehq/airbyte/issues/36967) mentioned that syncing 11M rows (5GB) took over an hour, despite having 8 CPU and 32GB of memory available. 3. In [another forum post](https://discuss.airbyte.io/t/2703), a user experienced syncs getting stuck at specific points, even with relatively small amounts of data. Your situation of 4K rows getting stuck for hours seems to align with these reported issues. The log message you're seeing (

Workload dbf246d7-8seg-4c20-b226-2d0fec5f6454_135_0_sync is pending

) suggests that the sync job is waiting to be processed, which could indicate resource constraints or scheduling issues. Here are a few potential reasons and suggestions based on the knowledge sources: 1. Resource allocation: Even though you have 8 CPUs and 32 GB of memory, Airbyte might not be utilizing all of it. Check your Airbyte configuration to ensure it's allowed to use the available resources. 2. Concurrent syncs: If you're running multiple syncs simultaneously, it could be causing resource contention. Try limiting the number of concurrent syncs. 3. Adaptive fetch size: As mentioned in the GitHub issue, the adaptive fetch size can significantly impact sync speed. You might want to investigate if this is affecting your sync performance. 4. Scheduler issues: Some users have reported problems with the Temporal Scheduler causing syncs to hang. Ensure you're using the latest version of Airbyte and check if there are any known issues with the scheduler in your version. 5. Connection or network issues: There could be intermittent connection issues between your EC2 instance and the source/destination databases. To troubleshoot further, you could try: 1. Monitoring resource usage during the sync to see if there are any bottlenecks. 2. Checking the full logs for any error messages or warnings. 3. Trying to sync a smaller subset of data to see if it completes successfully. 4. Verifying network connectivity and permissions between your EC2 instance, SQL Server, and Snowflake. If the problem persists, you might want to consider opening an issue on the [Airbyte GitHub repository](https://github.com/airbytehq/airbyte) with detailed logs and configuration information.