I have Airbyte set up to pipe data from HubSpot to Redshift yesterday. I decided to leave it running a sync overnight since there are 500k contact objects in HubSpot...but Airbyte only loaded like 50k in, and it took the whole night. Trying to figure out where I went wrong? • Infra: Kubernetes (EKS) • HubSpot API Tier: 150 calls per 10 seconds -- 500,000 calls daily (which we haven't even come close to) • Normalization setting:

Raw data (JSON)

Has anyone else experienced this? When we were using Stitch, it would load all 500k contacts in about 2 hours.

Based on the information provided in the extracts, there could be several reasons why your data sync from HubSpot to Redshift is taking longer than expected. Here are a few possibilities: 1. API Limitations: As mentioned in the [forum discussion](https://discuss.airbyte.io/t/2654), the HubSpot API has certain limitations that could affect the speed of data extraction. It's possible that the large volume of records is causing the sync to take longer. 2. Memory and Batch Size: In a [Slack conversation](https://airbytehq.slack.com/archives/C021JANJ6TY/p1675175911653049), it was suggested that increasing the memory of your server could help speed up the data load. The connectors use dynamic batch sizes, so increasing the memory of the source connector will make it consume more records each batch. 3. Airbyte Configuration: In another [Slack conversation](https://airbytehq.slack.com/archives/C021JANJ6TY/p1670479323992799), it was suggested that the sync speed could be affected by the configuration of your Airbyte server. You might want to check your Airbyte version and connector versions, and consider upgrading if necessary. 4. Normalization Process: As discussed in a [Slack conversation](https://airbytehq.slack.com/archives/C021JANJ6TY/p1669154392351759), the normalization process can take a significant amount of time, especially if you're using full refresh streams which recreate the tables each time. If you're still experiencing issues, you might want to check the sync logs for any errors or issues that could be slowing down the process. You can also consider reaching out to the Airbyte community on their [Slack channel](https://airbytehq.slack.com/archives/C021JANJ6TY) or [forum](https://discuss.airbyte.io) for further assistance.

Even when I do a

Full refresh - Overwrite

Sync mode for the contacts object, it loaded 36,454 contacts -- we have 552,969