Hello team, I'm migrating a connector from the low-code CDK to the python CDK in the goal of making a PR for a new connector we are using at my company. I have just one issue, in the low-code CDK, there's an option to inject the next page url, but the only examples I have found for the python CDK are injecting a new parameter in the request parameters. The API i'm ingesting from specifies the next page url in this param:

json_response["links"]["next"]

how can I use this url directly in the python CDK ?

hi Sami, have you tried seeting the page_token_request as RequestPath like in this example?

Hi, thanks for the answer but I moved out from using the low-code CDK, I need the answer for Python 🙂

Hi Sami, maybe you could overwrite the "path" function? In the next_page_token function you could get this next page url out of the response and return it and then in the path function (which has next_page_token as a parameter) you can use this next_page_token to define the path (assuming the base url stays the same). check the source: https://github.com/airbytehq/airbyte/blob/master/airbyte-cdk/python/airbyte_cdk/sources/streams/http/http.py