Hi, for performance and cost concerns, we use

incremental checkpointing

and configure

state.checkpoints.num-retained

parameter to retain 10 checkpoints only in our Flink applications. Although Flink keeps only 10 checkpoints, the number of files under

shared

folder keeps increasingly. To reduce the storage cost, it seems that the

CLAIM mode

mentioned in this doc is useful. We configure

execution.savepoint-restore-mode

to

CLAIM

but Flink still doesn’t delete files under the

shared

folder. I’m curious about the

CLAIM mode

only delete savepoints and checkpoints after the Flink app starts from using a new savepoint? If so, does anyone know whether there exists some way to limit the growth of files under the

shared

folder for checkpoints? Any comment is appreciated 🙏

@Chen-Che Huang Hello, do you have any findings in this topic? I'm also struggling with constantly growing shared directory

The issue has been solved in Flink 1.17. From release note

In 1.17, a REST API is provided so that users can manually trigger checkpoints with a self-defined checkpoint type while a job is running. For example, for a job running with incremental checkpoint, users can trigger a full checkpoint periodically or manually to break the incremental checkpoint chain to avoid referring to files from a long time ago.

We can break the incremental checkpoint chain periodically, and configure retention to keep checkpoint storage small.