I have observed it many times that few initial checkpoints after a job restart takes significantly longer time to complete than usual duration and unfortunately that breaches the checkpoint timeout limit, and hence leading to checkpoint failure. You can try

-Dexecution.checkpointing.timeout=15min

config to allow some more time for checkpoint to happen and see if it goes through.

default checkpoint timeout is

10 mins

, you can try with 2x or 3x of that.

there might be multiple reasons for this extended checkpoint duration during few initial checkpoint triggers • I assume that you have

aligned checkpoints mode

set to true (which is the default configuration). In this mode, an operator has to wait for barrier alignment from all input channels. Now, after restart, some of the operators might be overloaded compared to others based on the throughput difference across sources, and this skewness will lead to higher alignment time. Eventually this skewness will subside as operators catches up with the backlog and checkpoint duration will settle down to some stable value. • Second can be establishing initial connections with checkpoint store etc.