Lenin Mishra
12/15/2025, 1:56 AMkapa.ai
12/15/2025, 1:56 AMDatetimeBasedCursor with Unix epoch seconds by configuring the datetime format as %s (Unix timestamp in seconds). Airbyte explicitly supports this both in the YAML/low-code CDK and in the Connector Builder UI.[\[DatetimeBasedCursor ref\]](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/DatetimeBasedCursor)[\[Builder incremental\]](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/incremental-sync) ### If you’re using a declarative (manifest/YAML) connector In your DeclarativeStream, configure incremental_sync like this (adapt field names as needed): yaml incremental_sync: type: DatetimeBasedCursor cursor_field: "updatedAt" # your epoch-seconds field datetime_format: "%s" # cursor is Unix timestamp (seconds) cursor_granularity: "PT1S" # 1-second granularity start_datetime: datetime: "{{ config['start_date'] }}" datetime_format: "%Y-%m-%d %H:%M:%S" end_datetime: datetime: "{{ today_utc() }}" datetime_format: "%Y-%m-%d" step: "P1D" # window size (e.g. 1 day) start_time_option: field_name: "updated_after" # how your API expects the start time inject_into: "request_parameter" end_time_option: field_name: "updated_before" # how your API expects the end time inject_into: "request_parameter" This pattern is the same as the example used for Unix timestamps in the Delighted connector.[\[Forum UNIX timestamps\]](https://discuss.airbyte.io/t/3921) ### If you’re using the Connector Builder UI In the Incremental sync section for your stream: 1. Cursor field: set to updatedAt. 2. Cursor datetime formats (or “Datetime format” in older UI): add %s so Airbyte parses updatedAt as Unix seconds.[\[Builder incremental\]](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/incremental-sync) 3. Configure Start datetime and End datetime (usually “now”) in a human-readable format (e.g. %Y-%m-%dT%H:%M:%S). 4. Under Inject start/end time into outgoing HTTP request, map those times to your API’s query/body parameters (e.g. updated_after, updated_before). With %s as the cursor format, Airbyte will correctly track and advance the incremental state using your integer epoch updatedAt field.Lenin Mishra
12/15/2025, 2:31 AMkapa.ai
12/15/2025, 2:31 AMkapa.ai
12/15/2025, 2:31 AMupdatedAt as the cursor field (with %s as the datetime format). 4. Ensure the endpoint returns records sorted from newest to oldest across pages. Airbyte will: - Read pages in order, - Stop when it reaches records whose updatedAt is ≤ the stored cursor, - Save the latest updatedAt as state for the next sync. [[No filter mode](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] If the data is not sorted newest → oldest, the docs recommend not using incremental sync and instead doing full refreshes on a schedule. [[No filter mode](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)]Lenin Mishra
12/15/2025, 2:35 AMkapa.ai
12/15/2025, 2:35 AMkapa.ai
12/15/2025, 2:36 AMupdatedAt epoch field; you do not need to switch to an incrementing count cursor. From the docs, there are two incremental patterns in the Connector Builder: [[Builder incremental](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/incremental-sync)][[No filter mode](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] 1. Range mode (with time filtering) - You inject start/end datetimes into the request (query/body/header/path). - This is the part you can’t use because your API doesn’t accept time filters. 2. “No filter” mode (incremental sync without time filtering) - For APIs that do not allow filtering by date, but return a paginated feed sorted from newest to oldest. - You still configure a cursor field (your updatedAt in epoch seconds, using %s as the datetime format). - Airbyte keeps reading pages until it encounters records whose cursor value is ≤ the stored cursor, then stops and saves the latest cursor for the next sync. [[No filter mode](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] So in your case: - Keep using Datetime-based cursor on updatedAt (epoch seconds with %s). - Set API time filtering capabilities to “No filter”. - Make sure the endpoint returns data **newest → oldest across pages**; otherwise, the docs recommend disabling incremental and using full refresh instead. [[No filter mode](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)]Lenin Mishra
12/15/2025, 2:41 AMkapa.ai
12/15/2025, 2:41 AMkapa.ai
12/15/2025, 2:41 AMupdatedAt with %s) and pagination, and stops when it reaches records whose cursor ≤ the stored cutoff. [[No filter mode 1.8](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] However, the knowledge sources do not show: - Whether this option has been renamed, moved, or hidden in the latest Builder UI. - Where exactly in the current UI you should click to change it. So, based only on the docs: 1. Conceptually, you should be able to configure your stream as: - Cursor field: updatedAt - Cursor datetime format: %s (epoch seconds) [[Builder incremental (next)](https://docs.airbyte.com/platform/next/connector-development/connector-builder-ui/incremental-sync)] - API time filtering capabilities: “No filter” (so no start/end injection is required) [[No filter mode 1.8](https://docs.airbyte.com/platform/1.8/connector-development/connector-builder-ui/incremental-sync#incremental-sync-without-time-filtering)] 2. If you don’t see that option in your UI, the knowledge sources don’t explain why (version difference, feature flag, or UI change), so I can’t reliably tell you how to expose it from within the product. Given these limits, your next concrete steps (based on the docs) are: - Double‑check you’re in the Incremental sync configuration for the stream (not just pagination or schema). That’s where “API time filtering capabilities” is documented to live. [[Incremental sync 1.7](https://docs.airbyte.com/platform/1.7/connector-development/connector-builder-ui/incremental-sync)] - If the “API time filtering capabilities” / “No filter” control truly isn’t present, this is outside what the current docs cover. In that case, you’ll likely need to: - Either use a manifest/low‑code YAML connector where you can explicitly set is_data_feed / is_client_side_incremental on a DatetimeBasedCursor, which is the config‑file equivalent of “No filter”; [[DatetimeBasedCursor ref](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/DatetimeBasedCursor)] - Or contact Airbyte support / community with your Builder version and screenshots, since the docs don’t describe the UI state you’re seeing. So: the behavior you want is supported conceptually (datetime cursor with no time filters, using a feed), but the knowledge sources don’t contain enough information to explain why you can’t see the “API time filtering capabilities / No filter” option in your specific Builder UI.Lenin Mishra
12/15/2025, 2:46 AMkapa.ai
12/15/2025, 2:46 AMkapa.ai
12/15/2025, 2:47 AMupdatedAt (epoch seconds) as a datetime cursor in a low‑code YAML connector. Below is a template snippet that demonstrates how to wire a DatetimeBasedCursor to an updatedAt field expressed as Unix seconds, using an API key in a header and simple “single page” pagination. You will need to adapt the parts marked with comments (# TODO) to match the real Holded API (auth header name, pagination params, response field names, etc.), because those details are not in the knowledge sources. This pattern is based on: - DatetimeBasedCursor schema and properties (including cursor_field, cursor_datetime_formats, start_datetime, step, lookback_window, etc.). [[Datetime cursor ref](https://docs.airbyte.com/platform/1.8/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/DatetimeBasedCursor)] - How incremental syncs with datetime cursors work in low‑code connectors. [[Incremental syncs](https://docs.airbyte.com/platform/1.8/connector-development/config-based/understanding-the-yaml-file/incremental-syncs#datetimebasedcursor)] - Handling Unix timestamps via %s in cursor_datetime_formats, as shown in the Delighted example. [[Forum UNIX timestamps](https://discuss.airbyte.io/t/3921)] yaml version: "0.78.0" # TODO: set to the Airbyte spec version you are targeting definitions: base_requester: type: HttpRequester url_base: "<https://api.holded.com/api/invoicing/v1/>" authenticator: type: ApiKeyAuthenticator api_token: "{{ config['api_key'] }}" # TODO: confirm the correct header name for Holded; this is a guess inject_into: type: RequestOption field_name: "Authorization" inject_into: "header" contacts_stream: type: DeclarativeStream name: "contacts" primary_key: "id" # TODO: adjust to actual primary key field retriever: type: SimpleRetriever requester: $ref: "#/definitions/base_requester" path: "contacts" http_method: "GET" request_parameters: {} request_headers: {} paginator: type: NoPagination # TODO: replace with proper paginator if the API is paginated schema_loader: type: InlineSchemaLoader schema: type: object additionalProperties: true properties: id: type: string updatedAt: type: integer # TODO: add other fields as needed incremental_sync: type: DatetimeBasedCursor cursor_field: "updatedAt" # updatedAt is epoch seconds cursor_datetime_formats: - "%s" cursor_granularity: "PT1S" # earliest date to sync from, provided by user start_datetime: datetime: "{{ config['start_date'] }}" datetime_format: "%Y-%m-%dT%H:%M:%S" # end time is "now" in UTC end_datetime: datetime: "{{ today_utc() }}" datetime_format: "%Y-%m-%dT%H:%M:%S" step: "P1D" # 1‑day windows # No start_time_option / end_time_option here because the docs you provided # don’t show how Holded accepts time filters; this assumes a data‑feed style # where the API is read fully and the cursor is applied client‑side. is_client_side_incremental: true streams: - $ref: "#/definitions/contacts_stream" check: type: CheckStream stream_names: - "contacts" spec: connectionSpecification: type: object required: - api_key - start_date properties: api_key: type: string title: API Key description: "Holded API key" airbyte_secret: true start_date: type: string title: Start date description: "Earliest updatedAt to sync from (e.g. 2023-01-01T00:00:00)" Key points you’ll likely need to verify/adjust based on the real API (not covered in the knowledge sources): - Auth header: whether Holded expects Authorization: XYZ, X-API-KEY: XYZ, or something else. - Pagination: if /contacts is paginated, replace NoPagination with the appropriate paginator (e.g. cursor or page/limit). - Time filtering: if the API does support updatedAt filters (e.g. updatedAtFrom / updatedAtTo), you can add start_time_option and end_time_option to the DatetimeBasedCursor as shown in the Unix‑timestamp example from the forum. [[Forum UNIX timestamps](https://discuss.airbyte.io/t/3921)] - Schema: adjust primary_key and properties to match the actual response. Because the Holded‑specific details are not in the provided docs, please treat this as a starting template rather than a drop‑in manifest.Lenin Mishra
12/15/2025, 2:56 AMkapa.ai
12/15/2025, 2:56 AMkapa.ai
12/15/2025, 2:56 AMIncrementingCountCursor component. This cursor assumes a monotonically increasing integer field (e.g. id, offset, etc.) and uses it as the bookmark for incremental syncs. [[Incrementing cursor](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/IncrementingCountCursor)] ### 1. When to use IncrementingCountCursor Use it when: - Your records have a top‑level integer field that always increases (e.g. id, sequence, offset). - The API returns records **sorted ascending by that field**; the docs explicitly require this to avoid data loss. [[Incrementing cursor](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/IncrementingCountCursor)] ### 2. Core configuration fields From the YAML reference: [[Incrementing cursor](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/IncrementingCountCursor)] - cursor_field (string, required) Top‑level field used as the bookmark (e.g. id). - allow_catalog_defined_cursor_field (boolean, optional) If true, lets users override cursor_field from the connection catalog. - start_value (string or integer, required) Earliest value to sync from (e.g. 0 or {{ config['start_value'] }}). - start_value_option (RequestOption, optional) How to send the start_value to the API (e.g. as a query parameter). ### 3. Example: using id as an incrementing cursor Below is a minimal example of wiring IncrementingCountCursor into a declarative stream: yaml definitions: my_stream: type: DeclarativeStream name: my_stream primary_key: id retriever: type: SimpleRetriever requester: type: HttpRequester url_base: "<https://api.example.com/>" path: "items" http_method: "GET" paginator: type: NoPagination schema_loader: type: InlineSchemaLoader schema: type: object additionalProperties: true properties: id: type: integer incremental_sync: type: IncrementingCountCursor cursor_field: "id" allow_catalog_defined_cursor_field: false start_value: 0 # Optional: if the API accepts a "min_id" parameter start_value_option: type: RequestOption field_name: "min_id" inject_into: "request_parameter" Key points from the docs applied here: [[Incrementing cursor](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#/definitions/IncrementingCountCursor)] - cursor_field is top‑level and integer. - start_value defines the first record to pull. - start_value_option (optional) shows how you’d inject that into the request if the API supports it. If your API doesn’t accept a filter parameter, you can omit `start_value_option`; Airbyte will still track state client‑side, but you must ensure the API always returns records in ascending order by cursor_field to avoid missing or duplicating records.Lenin Mishra
12/15/2025, 3:00 AMkapa.ai
12/15/2025, 3:00 AMkapa.ai
12/15/2025, 3:01 AM["readable_updated_at"]. 4. Value: a Jinja expression that transforms your epoch field. If your epoch is already a string or integer field on the record (e.g. record['updatedAt']), you can follow the same pattern as the hashing example, but using a date conversion filter if available in your environment. The docs show the general pattern for using filters like: jinja {{ record['from']['name'] | hash('md5') }} for hashing; you’d replace hash('md5') with the appropriate date conversion filter supported by your runtime. [[Adding fields](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/record-processing#adding-fields)] However, the provided knowledge sources do not document a built‑in Jinja filter that converts epoch seconds to a formatted date (only hash is explicitly shown), so I cannot give you a guaranteed‑working exact expression for epoch→date in the Builder UI. --- ## 2. In a low‑code (YAML) declarative manifest Use the AddFields transformation with an AddedFieldDefinition. [[Record selector transformations](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/record-selector#transformations)] Example structure: yaml transformations: - type: AddFields fields: - path: ["readable_updated_at"] value: "{{ record['updatedAt'] }}" - path is where the new field will be added (top‑level or nested). [[Record selector transformations](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/record-selector#transformations)] - value is a string expression; you can reference the original epoch field via record['updatedAt']. [[AddedFieldDefinition](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#addedfielddefinition)] Again, the docs show how to copy or hash existing fields, but they do not document a built‑in function to convert epoch seconds to a formatted date directly in YAML (e.g. no example of | to_datetime or similar), so I can’t provide a confirmed expression that will both parse epoch and format it. --- ### Summary - You can add a new field derived from your epoch field using Add / AddFields transformations in both the Builder UI and YAML. [[Adding fields](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/record-processing#adding-fields); [Record selector transformations](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/record-selector#transformations)] - The knowledge sources show how to: - Reference existing fields (record[...]) and - Apply filters like hash('md5'). [[Adding fields](https://docs.airbyte.com/platform/connector-development/connector-builder-ui/record-processing#adding-fields); [AddedFieldDefinition](https://docs.airbyte.com/platform/connector-development/config-based/understanding-the-yaml-file/reference#addedfielddefinition)] - They do not explicitly document a built‑in Jinja filter for converting epoch seconds to a human‑readable date, so I cannot give a fully certain epoch→date expression; you may need to: - Do the epoch→date conversion downstream (e.g. in your warehouse), or - Implement a custom Python CDK connector and use a custom transform in code. [[Custom schema transform](https://docs.airbyte.com/platform/connector-development/cdk-python/schemas#custom-schema-type-transformation)]