Trying to understand how to implement checkpoint in custom source reader. My question is how do I know what all events are processed by last step and what to checkpoint?
I read about event barriers. Let's say we processing data A1, A2, A3, A4, A5, and checkpoint barriers B are inserted randomly.
And it got processed as below
A1, A2, B1, A3, A4, B2, A5
At any point if checkpoint on source function is called then how do I know which all events are processed by last step ? Is it A2 or A4 or so on.