Hi... Is there any ongoing work for giving savepoint as an additional output of a BATCH execution job?
d
David Anderson
03/10/2023, 3:23 AM
The snapshot mechanism used for checkpoints and savepoints in streaming jobs isn't very compatible with BATCH mode. I think that's unlikely to ever happen.
What's the motivation behind your question -- are you wishing for a different recovery/restart mechanism?
k
Kiran Shivappa Japannavar
03/11/2023, 2:50 PM
The batch is scheduled to run on a daily basis. So when the next day's run happens I need the state of the previous day's run to apply certain business logic. Also, the job does an incremental read on a daily level