Hi guys,
I was working in a fix of Zendesk source. The main reason for it was the slow process to retrieve satisfaction_ratings, which has tons of records there (~150k).
I was looking the code itself, and I notice that it calculates the number of pages based on total records, which take so long to cover all records, even if we already imported it before, and just looking for the new ones. This process creates more than 1k requests every run (and a lot of time). In addition, this approach starts the retry strategy, increasing the time to finish the process.
The Zendesk documentation already have this alternative based on
next_page
.
I’ve just created a custom version of the connector, to speed up our ingestion process.
I’m thinking if it makes sense to suggest this change to the main repository. Is this some reason to use that approach instead of this one?