This message was deleted.

The

HttpStream

class in Airbyte's CDK (Connector Development Kit) is used to facilitate reading data from HTTP endpoints. It contains built-in functions or helpers for authentication, pagination, handling rate limiting or transient errors, and other useful functionality. To populate the

HttpStream

class, you need to provide it with a few inputs: 1. The URL base and path of the endpoint you'd like to hit. 2. How to parse the response from the API. 3. How to perform pagination. Optionally, you can provide additional inputs to customize requests: - Request parameters and headers. - How to recognize rate limit errors, and how long to wait. - HTTP method and request body if applicable. - Configure exponential backoff policy. Here are the key methods and properties you need to define: - `url_base`: This property should point to the base URL of the API endpoint you want to hit. For example, in the Webflow source connector, the

url_base

is defined to point to

<https://api.webflow.com>

. - `path`: This is an abstract function that you need to implement. It should return the path of the specific API resource you want to access. - `parse_response`: This function instructs the stream how to parse the API response. It should return an

Iterable

, whose elements are each later transformed into an

AirbyteRecordMessage

. - `request_params`: This method returns the parameters for the API request. By default, it returns an empty dictionary, but you can override it in subclasses to specify the parameters you need. - `authenticator`: This method is used for authentication. You can override it to fix any issues or bugs in the authentication process. - `schemas`: The

HTTPStream

must describe the schema of the records it outputs using JsonSchema. You can do this by placing a

.json

file per stream in the

schemas

directory in the generated python module. The name of the

.json

file must match the lower snake case name of the corresponding Stream. For more details, you can refer to the [HTTP-API-based Connectors](https://docs.airbyte.com/connector-development/cdk-python/http-streams) section in the Airbyte documentation.