< kapa ai> how to apply a error handling on the method self Airbyte #replication-ask-ai

<@U04S8PLJWMT> how to apply a error handling on th...

Matheus Dantas

08/18/2026, 1:45 PM

@kapa.ai how to apply a error handling on the method self._http_client.send_request to not try again if the return code is 429. I want to apply a specific log message, providing details about the return header of the request with the information returned

kapa.ai

08/18/2026, 1:45 PM

The knowledge sources do not contain enough information to give you a precise answer about customizing

self._http_client.send_request

to skip retries on 429 and log header details, since the sources focus on retrying 429s rather than suppressing retries for them. However, based on what the sources do show about Airbyte's error handling framework, I can give you a well-grounded approach. Handling 429 Without Retrying (Custom ErrorHandler) The standard pattern in Airbyte's Python CDK is to implement a custom

ErrorHandler

by overriding

interpret_response

. You can return

ResponseAction.FAIL

(or

ResponseAction.IGNORE

) for 429s and log the response headers yourself. Here is an example derived from the patterns shown in the knowledge sources (e.g.,

ShopifyErrorHandler

and

GithubStreamABCErrorHandler

Copy code

import logging
from typing import Optional, Union

import requests
from airbyte_cdk.models import FailureType
from airbyte_cdk.sources.streams.http.error_handlers import ErrorHandler, ErrorResolution, ResponseAction

logger = logging.getLogger("airbyte")


class NoRetryOn429ErrorHandler(ErrorHandler):

    @property
    def max_retries(self) -> Optional[int]:
        return 0  # or keep at 5 for other errors

    @property
    def max_time(self) -> Optional[int]:
        return 60

    def interpret_response(
        self, response: Optional[Union[requests.Response, Exception]]
    ) -> ErrorResolution:
        if isinstance(response, requests.Response):
            if response.status_code == 429:
                # Extract and log all relevant headers
                retry_after = response.headers.get("Retry-After", "not provided")
                rate_limit_limit = response.headers.get("X-RateLimit-Limit", "not provided")
                rate_limit_remaining = response.headers.get("X-RateLimit-Remaining", "not provided")
                rate_limit_reset = response.headers.get("X-RateLimit-Reset", "not provided")

                logger.warning(
                    f"Rate limit hit (429). Will NOT retry. "
                    f"Response headers: "
                    f"Retry-After={retry_after}, "
                    f"X-RateLimit-Limit={rate_limit_limit}, "
                    f"X-RateLimit-Remaining={rate_limit_remaining}, "
                    f"X-RateLimit-Reset={rate_limit_reset}. "
                    f"Response body: {response.text}"
                )

                return ErrorResolution(
                    response_action=ResponseAction.FAIL,
                    failure_type=FailureType.transient_error,
                    error_message=(
                        f"Request was rate limited (HTTP 429) and will not be retried. "
                        f"Retry-After: {retry_after}"
                    ),
                )

            if response.ok:
                return ErrorResolution(ResponseAction.SUCCESS, None, None)

        # Delegate everything else to default behavior
        return ErrorResolution(ResponseAction.RETRY, FailureType.transient_error, "Unexpected error, retrying.")

Key points from the knowledge sources: • ResponseAction.FAIL
stops the sync immediately without retrying. [Response Filters] • ResponseAction.IGNORE
skips the current response and continues the sync without retrying. [Response Filters] • The

interpret_response

method is the standard place to inspect

response.headers

and log custom messages, as seen in the GitHub connector's error handler. [GitHub error handler] • The Shopify connector shows the same pattern for returning

ErrorResolution

based on status codes. [Shopify error handler] If using YAML / Connector Builder If you are using the declarative YAML framework instead, you can configure a response filter with

action: FAIL

for HTTP code 429:

Copy code

requester:
  error_handler:
    response_filters:
      - http_codes: [429]
        action: FAIL
        error_message: "Rate limited (429). Headers: {{ headers }}"

[YAML error handling] > Note: The knowledge sources do not explicitly document the internal API of

self._http_client.send_request

or how to hook into it directly. The approach above targets the

ErrorHandler

abstraction layer, which is the recommended and documented way to control retry behavior in Airbyte connectors. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

6 Views

Open in Slack

Previous Next