https://linen.dev logo
#connector-development
Title
# connector-development
e

Ethan Veres

03/15/2022, 2:36 PM
@Sherif Nada let me know if that all makes sense!
a

Augustin Lafanechere (Airbyte)

03/16/2022, 10:11 AM
Hey @Ethan Veres, I think the key here is to implement a
stream_slices
method on
IncrementalLeverHiringStream
to replace interval-based checkpointing by stream slices + enable the use of cache. I drafted an untested example, let me know if it makes sense.
e

Ethan Veres

03/16/2022, 2:46 PM
Thanks @Augustin Lafanechere (Airbyte)! Would calling
opportunities_stream.read_records(sync_mode=SyncMode.incremental ...
with incremental affect the opportunities saved state in the db?
And I guess a few other questions - 1. There’s no
updatedAt
cursor field in the opportunity child streams, so how can I get the incremental state for child streams saved?
a

Augustin Lafanechere (Airbyte)

03/18/2022, 1:11 PM
Thanks @Augustin Lafanechere (Airbyte)! Would calling
opportunities_stream.read_records(sync_mode=SyncMode.incremental ...
with incremental affect the opportunities saved state in the db?
If these records are read from the cache I don't think it will affect the state.
And I guess a few other questions -
1. There’s no
updatedAt
cursor field in the opportunity child streams, so how can I get the incremental state for child streams saved?
Do you think the opportunity_id could be used as a cursor field for these child streams?
3 Views