I'm trying out the new Airbyte Connector builder UI. The API endpoint is https://chargify.com/transactions.json?per_page=200&direction=asc&order_by=id&since_id=XXX (GET), and a reduced version of the JSON payload from the API is shown below. I've configured the connection as Cursor Pagination with a custom next page cursor. I haven't figured out the magical combination of Record Selector and Cursor Value, and I've tried about 100. What I'm looking to do is grab the

id

value from the last fetched record, and use that in the request parameter

since_id=

. I think part of the problem is that the JSON is formatted kind of funny, such that the JSONPath to get to the id value is difficult. Anybody have suggestions?

@Dan Cook try

{{ last_records[-1]['id'] }}

as the cursor value

I've tried that variation but it doesn't result in getting 5 test pages back. With other Cursor Value variations at least I get 5 pages, isn't that a sign of progress?

try recordselector=

transaction

and the cursor value i suggested above?

If I try

transaction

or

"transaction"

as the record selector I get an error message:

list object has no element -1

talked with dan offline, solution was

record_selector=[*, transaction]

and

cursor_value={{ last_records[-1]['id'] }}

I've tried several other tools with the API and Airbyte as of today is the only one that I've been able to paginate properly through. So well done Airbyte team! As of now the last result on page X is the first result on page X+1. Mathematically it's like I need to set the cursor using

>= id + 1

rather than

>= id

, but I'm not sure how to accomplish that. Never mind, I figured it out:

{{ last_records[-1]['id'] +1 }}