https://linen.dev logo
w

will

01/28/2022, 11:33 AM
Hi, I am trying to build a python source (using python source template) and was successful in implementing a full refresh sync. However, I am unable to add incremental feature to my stream(incremental option does not show in airbyte UI). I have done the following but with no success: -> I am passing the given values while creating a stream during the discover function execution.
"supported_sync_modes": [
"full_refresh",
"incremental"
],
"source_defined_cursor": True,
"default_cursor_field":["json_data","properties","updated_at"],
"source_defined_primary_key": ["json_data","properties","_id"],
Am I missing something?
s

Siddharth Putuvely

01/28/2022, 1:24 PM
Hi @Siddharth Putuvely, you need to declare a
cursor_field
attribute to your stream's class to make it discovered as an incremental stream by Airbyte. You'll also have to implement some logic around state management and stream slices. Please check this documentation about incremental stream implementation: https://docs.airbyte.com/connector-development/cdk-python/incremental-stream
8 Views