<@U04S8PLJWMT> i wanted to know if airbyte allows ...
# replication-ask-ai
h
@kapa.ai i wanted to know if airbyte allows composite cursor keys also ? i know it allows composite primary key for sure. i am facing an issue where i am getting-> Only top-level cursors are supported. and my streams config looks like this
k
Looking at your stream config in the image, I can see your
cursorField
is set to
["Added_Time", "Modified_Time"]
— this is being interpreted as a nested cursor path (i.e.,
Added_Time.Modified_Time
), not a composite cursor with two separate fields. Composite Cursor Keys: Not Supported Airbyte does not support composite cursor keys (multiple independent fields used together as a cursor). The
cursor_field
array in the protocol is used to define a path to a single nested field, not multiple cursor fields. [Airbyte Protocol - Cursor] For example: •
["metadata", "updated_at"]
→ means the cursor is at
record.metadata.updated_at
(a nested path to one field) • It does not mean "use both
metadata
and
updated_at
as cursors" Why You're Getting "Only top-level cursors are supported" The normalization layer explicitly raises an error for nested cursor fields. From the source code,
get_cursor_field
raises:
Copy code
ValueError: Unsupported nested cursor field {field1}.{field2} for stream {stream_name}
when
len(self.cursor_field) > 1
. [normalization source] This has also been confirmed in community discussions — using multiple fields as a cursor field breaks normalization and is not currently supported. [forum post] [forum post] What You Can Do 1. Use a single cursor field — Pick either
Added_Time
or
Modified_Time
as your cursor (whichever is more reliable for tracking new/updated records). 2. Create a synthetic combined field — If you control the source connector, you can combine the two fields into a single top-level field (e.g., a concatenated or max value) and use that as the cursor. 3. Use an "Add Field" transformation — If the cursor is nested, use an "Add Field" transformation to promote it to the top level, then use that top-level field as the cursor. [Connector Builder docs] 4. Fall back to Full Refresh — If no single reliable cursor exists, full refresh is a safe fallback until composite cursors are supported. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Looking at your stream config in the attached image, I can see you have
cursorField: ["Added_Time", "Modified_Time"]
— that's two separate fields being used as a composite cursor. Airbyte does NOT support composite cursor fields. The cursor field must be a single top-level field. Here's what the knowledge sources confirm: 1. Only a single cursor field is supported. Using multiple fields as a cursor is not currently possible in Airbyte. [composite cursor forum] 2. Nested cursor fields are also unsupported. The cursor must be at the top level of the record — not nested inside objects. [incremental sync docs] 3. The normalization layer explicitly raises an error for cursor fields with more than one element in the path array, as seen in the source codeelse raise ValueError(f"Unsupported nested cursor field {'.'.join(self.cursor_field)} for stream {self.stream_name}")This is why you're seeing the "Only top-level cursors are supported" error. [stream_processor.py] What you can do instead: • Use only one of the two fields (e.g.,
Added_Time
or
Modified_Time
) as your single cursor field. • If neither field alone is sufficient, consider switching to Full Refresh sync mode as a workaround until Airbyte adds support for this. [cursor forum] So while composite primary keys are supported, composite cursor fields are not. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.