Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi all,
We have a `Hubspot -&gt; S3` connector with the following configs and we’d like to understand why it takes a bunch of time to run a sync and whether it can be sped up in any way.

For the deployment we are using the *helm chart* with the following resource settings for jobs (this is not reflected in the destination definition which is weird, however the `source-*` and `destination-*`  pods are using these limits):
```global:
  jobs:
    resources:
      requests:
        cpu: "200m"
        memory: "4Gi"
      limits:
        cpu: "200m"
        memory: "4Gi"```
*Airbyte version*: _`0.40.17`_

*Source definition:*
```{
  "sourceDefinitionId": "36c891d9-4bd9-43ac-bad2-10e12756272c",
  "name": "HubSpot",
  "dockerRepository": "airbyte/source-hubspot",
  "dockerImageTag": "0.2.3",
  "documentationUrl": "<https://docs.airbyte.io/integrations/sources/hubspot>",
  "protocolVersion": "0.2.0",
  "releaseStage": "generally_available"
}```
*Destination definition:*
```{
  "destinationDefinitionId": "4816b78f-1489-44c1-9060-4b19d5fa9362",
  "name": "S3",
  "dockerRepository": "airbyte/destination-s3",
  "dockerImageTag": "0.3.17",
  "documentationUrl": "<https://docs.airbyte.com/integrations/destinations/s3>",
  "protocolVersion": "0.2.0",
  "releaseStage": "generally_available",
  "resourceRequirements": {
    "jobSpecific": [
      {
        "jobType": "sync",
        "resourceRequirements": {
          "memory_request": "1Gi",
          "memory_limit": "1Gi"
        }
      }
    ]
  }
}```
*Source:*
```{
  "sourceDefinitionId": "36c891d9-4bd9-43ac-bad2-10e12756272c",
  "sourceId": "457f3db8-6ce1-41be-9ecb-7ef9a724c88b",
  "workspaceId": "2b94a777-1e5e-4381-af9f-21582ecce5c7",
  "connectionConfiguration": {
    "start_date": "2022-11-15T12:00:00Z",
    "credentials": {
      "access_token": "**********",
      "credentials_title": "Private App Credentials"
    }
  },
  "name": "hubspot_test",
  "sourceName": "HubSpot"
}```
*Destination:*
```{
  "destinationDefinitionId": "4816b78f-1489-44c1-9060-4b19d5fa9362",
  "destinationId": "033a010b-ff8e-4eb3-9ee6-6505a6c42d00",
  "workspaceId": "2b94a777-1e5e-4381-af9f-21582ecce5c7",
  "connectionConfiguration": {
    "format": {
      "compression": {
        "compression_type": "No Compression"
      },
      "format_type": "JSONL"
    },
    "s3_endpoint": "",
    "access_key_id": "**********",
    "s3_bucket_name": "****",
    "s3_bucket_path": "****",
    "s3_bucket_region": "****",
    "secret_access_key": "**********"
  },
  "name": "hubspot_s3",
  "destinationName": "S3"
}```
Do you know what can cause the pulling of only `3 MBs` of data to take `~3 hours`? Also do you have any recommendations on how to handle this? Many thanks :pray:

Hello Gergely Lendvai, it's been a while without an update from us. Are you still having problems or did you find a solution?

Hi, I haven't found any solutions yet, do you have any recommendations or suggestions we could use? <@U02HJQXRBD2>