Question regarding

FlinkKinesisConsumer

: as it doesn’t have committed offsets like kafka, after a restart (or new deploy), i’d love to see flink to continue reading data from the point he left and not from

latest

or

timestamp

. Can anyone share how they approached this issue ? I’m sure that i’m not the only one with this usecase ? I don’t see flink starting from the checkpoint/savepoint.

I believe this is explained at https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kinesis/#fault-tolerance-for-exactly-once-user-defined-state-update-semantics

well, looks like i’m missing something. I do have checkpointing enabled, but from what i see, first checkpoint is triggered before (or restoring from them, we use s3 as a storage) consumer starts reading from kinesis

I do think that doesn’t matter. If you enable checkpoint, you should have (some) resilience. Which can be exactly once, at least once, at most once etc

do i have a way of enforcing restoring from s3 checkpoint before

FlinkKinesisConsumer

starts pulling data ? in my usecase it’s a source of data.

If you restart from a checkpoint, that is always done IIRC. It’s default behavior