I was able to setup checkpointing in my local flin...
# troubleshooting
j
I was able to setup checkpointing in my local flink installation by adding these properties in my
conf/flink-conf.yaml
file
Copy code
execution.checkpointing.interval: 1min
state.backend: filesystem
state.checkpoints.dir: <s3://my-s3-bucket/checkpoints/>
However adding the same properties in my flink cluster running in kubernetes does not enable checkpointing as seen in these screenshots. The flink webui running on localhost has the correct checkpointing interval(1min) while the one running on EKS has periodic checkpoints disabled. What should I check?
g
You can check the pods as a starting point to see if the configurations are set in the configmaps
j
yes i already did that
i can also see the
execution.checkpointing.interval
property in the jobmanager configuration page in the webui,
g
And the logs don’t indicate any s3 connection issues, correct?
j
yes. no exceptions in the logs. i also tried to check the s3 connectivity within the node/pod.
o
It seems like it defaulted over to JobManagerCheckpointingStorage. Is the s3 plugin enabled on the EKS-based flink cluster?