Hi everyone. I am currently running a streaming pipeline using kafka->flink(beam) via flink k8s operator->clickhouse in k8s. I am using hadoop as my storage dir for checkpoint and savepoints. This setup works well. However, I noticed that the checkpoint size slowly increases after running it for day. The messages being processed is around 200 messages/second with about 2k per message. I set

state.checkpoints.num-retained: "5"

hoping that the size won't grow any more beyond a certain size but I am still seeing a increasing trend when I view metrics from grafana. Is there a way to cleanup the checkpoint files which are no longer relevant for checkpoint recovery?

If you’re using Flink 1.15 or higher, I would recommend reading https://flink.apache.org/2022/05/06/restore-modes.html - Then you can know for sure if either you can cleanup files or if they are owned by Flink and should be left alone