Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi everyone, I am building a custom connector with IncrementalMixin slices, but am being throttled in the children streams with too many parent ids.  Could someone suggest a way to, or point me to an example of the best way to limit concurrent requests?

Hi! Based on the information extracts provided, the CDK will conduct exponential backoff on HTTP code 429 and any 5XX exceptions by default, and fail after 5 tries. Retries are governed by the `should_retry` and the `backoff_time` methods. You can override these methods to customize the retry behavior. Here's an [example from the Slack API](<https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-slack/source_slack/source.py#L72>).  However, there isn't enough information in the extracts to provide a specific solution for limiting concurrent requests in your custom connector. I recommend reviewing the [Airbyte documentation](<https://docs.airbyte.com/connector-development/cdk-python>) and [Airbyte forum](<https://discuss.airbyte.io>) for more guidance on custom connector development.