It sounds like you're experiencing performance issues with syncing a large dataset from Oracle to Postgres using Airbyte. Based on the knowledge sources provided, here are some actionable steps and insights to help you optimize the sync process: 1. Increase Resource Allocation: - Ensure that you have allocated sufficient resources to both the source and destination connectors. Increasing the CPU and memory limits can help improve performance. For example, in one of the cases, increasing the connection’s CPUs and memory to the maximum available resources showed some improvement ([source](https://discuss.airbyte.io/t/3284)). 2. Adjust Fetch Size: - The fetch size can significantly impact the sync speed. You can try adjusting the

DEFAULT_FETCH_SIZE

to find an optimal value that works for your setup. In one instance, adjusting the fetch size showed a significant impact on sync speed ([source](https://github.com/airbytehq/airbyte/issues/36967)). 3. Use Incremental Sync: - If possible, use incremental sync instead of full sync. This can drastically reduce the amount of data being processed in each sync operation. For example, in a Postgres to Postgres sync, using incremental deduplication helped manage large datasets more efficiently ([source](https://discuss.airbyte.io/t/3111)). 4. Optimize Normalization: - If normalization is taking a long time, consider whether it can be done incrementally or if there are ways to optimize it. In one case, normalization of records took more than 10 hours, and the user was looking for ways to make it incremental ([source](https://discuss.airbyte.io/t/3111)). 5. Monitor and Adjust Batch Size: - Decreasing the batch size can sometimes help with performance, especially if you are running into memory issues. However, this can be a trade-off as smaller batch sizes might increase the number of operations. Experiment with different batch sizes to find the optimal setting ([source](https://github.com/airbytehq/airbyte/discussions/28726)). 6. Check for Bottlenecks: - Ensure that there are no bottlenecks in your network or database configurations. Sometimes, the issue might not be with Airbyte but with the underlying infrastructure or network latency. 7. Use ThreadPools: - If you are using a custom connector, consider using ThreadPools to parallelize the data fetching process. This approach showed some improvement in performance in one of the tests ([source](https://discuss.airbyte.io/t/3284)). By following these steps, you should be able to optimize the sync process and reduce the time it takes to write data to Postgres. If you continue to experience issues, you might want to reach out to the Airbyte community for further assistance or check for any updates or performance improvements in newer versions of Airbyte. For more detailed discussions and potential solutions, you can refer to the following sources: - [Source 1](https://discuss.airbyte.io/t/3284) - [Source 2](https://github.com/airbytehq/airbyte/issues/36967) - [Source 3](https://discuss.airbyte.io/t/3111) - [Source 4](https://github.com/airbytehq/airbyte/discussions/28726)