Hello, We're having some issues where continuo...
# troubleshooting
a
Hello, We're having some issues where continuously running flink jobs restart and begin processing from hours ago. The events are pulled off a Kafka topic. We write checkpoints to s3 every 5 minutes - but when we enter into these failures, and all task managers exit and restart, the recovering the jobs load the HA and pick a checkpoint which is often hours old. Not the latest. What are we doing wrong? We'd like to reduce reprocessing any events, and really keep backpressure under 30 minutes ideally, definitely under an hour.
m
Hello Aaron, Please check the Flink Web UI and JobManager logs for checkpoint issues: • In the Web UI → Checkpoints: what’s the latest completed checkpoint timestamp, and are there failed/in-progress checkpoints? • Check the job’s restart/restore history — which checkpoint was used to restore? • In JobManager logs around those checkpoint times, look for errors like “could not persist checkpoint”, write/permission errors, or S3/metadata exceptions.
🙇🏻 1