Hi everyone, I am trying to sync a <products endpo...
# help-connector-development
l
Hi everyone, I am trying to sync a products endpoint that doesnt have a datetime column that I can use for incremental sync. Can I enable it to be incremental or not?
k
You generally cannot use true incremental sync in Airbyte when: - The API does not return any “last updated / last modified” field in the records, and - You also cannot filter or request records by such a cursor field. Airbyte’s incremental mode in the Connector Builder requires: - A top‑level date/time field in each record used as a cursor field, and - The ability to filter/request records by that field. [[Incremental sync basics](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/incremental-sync)] If the products endpoint doesn’t expose such a field and doesn’t support time-based filtering, you have two main options: 1. Full refresh with stream slices Use Full Refresh and slice by something like date ranges (if the API supports query params for date) or other dimensions (e.g., pages). This is explicitly recommended when there is no modification date / cursor suitable for incremental sync. [[Forum 1710 answer](https://discuss.airbyte.io/t/1710)] 2. Custom overlapping-window logic in a custom CDK connector In Python CDK, for some APIs that accept a
lastmodified
parameter but don’t return it in the data, you can store that value in state and use overlapping time slices (lookback window) to reduce risk of missed changes. [[Forum 344 answer](https://discuss.airbyte.io/t/344)] However, this still relies on the API supporting a time filter; if the products endpoint doesn’t allow any such filter, this approach won’t work either. The Connector Builder’s built‑in “No filter” incremental mode only works if the API returns a cursor field (e.g., created/updated datetime) and the feed is strictly ordered (newest → oldest). Without any datetime field in the records, you also can’t use this. [[No filter incremental](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] So, based on the docs and forum guidance: - If the products endpoint truly has no datetime/cursor field in the returned records and no way to filter by time, you should use Full Refresh (optionally with stream slices), not incremental.