Ignacio Penissi
10/25/2023, 6:07 PM<http://example.com/API/v1/orgid/schedule/<year>/r12|example.com/API/v1/orgid/schedule/<year>/r12>
In this structure, I need to dynamically inject the "year" parameter as part of the URL path. The challenge is that the next page token does not appear in the API response. Instead, we must manually increment the previously-used page token if the API response contains data. When no more data is present, we can assume there are no additional pages to retrieve.
My initial attempt to define the connector pagination in the manifest.yaml file is as follows:
paginator:
type: DefaultPaginator
pagination_strategy:
type: CustomPaginationStrategy
class_name: "source_example.my_pagination_strategy.CustomYearlyPaginationStrategy"
year: "{{ config['year'] }}"
However, I would prefer to use a non-custom paginator if possible to simplify the implementation. If that's not an option, I would appreciate guidance on how to implement the custom pagination strategy to achieve this dynamic path change based on the "year" parameter.
Any help or examples on how to structure the manifest.yaml file or suggestions on using an existing paginator for this unique use case would be greatly appreciated.kapa.ai
10/25/2023, 6:09 PMIgnacio Penissi
10/25/2023, 6:11 PMnext_page_token in the manifest.yaml file to achieve this dynamic path change based on the "year" parameter?Ignacio Penissi
10/26/2023, 12:19 PMnext_page_token in the manifest.yaml file to achieve this dynamic path change based on the "year" parameter?