hi team, can someone help me with my understanding...
# connector-development
g
hi team, can someone help me with my understanding of incremental configuring in all airbyte connectors. incremental stream obtains the last updated date for a record and sets this as the
cursor_field
so that on the next sync, the API will pull records after this date. however some connectors i’ve looked at all appear to pull records for on or after this date. meaning that it will always pull the last record synced again, causing duplicate records to appear. this is the case for
zendesk-support
,
zendesk-talk
,
survey-monkey
and i imagine others too. given no other users are complaining about this, i’m wondering if i’m misunderstanding some fundamental aspect of the incremental stream. posted additional details in the discussion forum if you’re interested