Based on our real data test, airbyte is quite strong ETL tool for oracle to Kafka. 👍 . May I know what is the mechanism of periodically sync between the source and destination(e.g. every 30 minutes), sometime the table may not have datetime column which may indicate the current day. Thanks, the outstanding airbyte team.

Hello @Timothy Tian Yang can you reframe your question or give a concrete example of your problem?

Hi @[DEPRECATED] Marcos Marx, we like to choose Sync frequency when setting up the new connection between source and destination, and choose sync mode to incremental append, may I know some best practice for how to set up cursor field? Thanks

Hi @Timothy Tian Yang the cursor field is made to identify records that will belong to the same incremental sync. According to our doc:

The cursor is how sources track which records are new or updated since the last sync.

A "cursor field" is the field that is used as a comparable for making this determinations.

If a configuration requires a cursor field, it requires an array of strings that serves as a path to the desired field. e.g. if the structure of a stream is

{ value: 2, metadata: { updated_at: 2020-11-01 } }

the

default_cursor_field

might be

["metadata", "updated_at"]

.

You'll find more details about this here or here.