Hello everyone, We have a stateful flink app, for which the checkpoint size is continuously growing. All the states have TTL config and manual cleanup mechanism. Everytime the app gets restarted due to a stateful upgrade, the checkpoint size drops to normal then again continues to grow overtime. (PFA screenshot) Need help to understand how can restoring from the exact same checkpoint result in smaller size checkpoint Has anyone faced a similar issue or any suggestion how we can further debug this?

Are you using incremental checkpoints with RocksDB?

We haven't enabled incremental checkpointing

If you're using rocksdb, there's a property that tells it to do periodic clean ups.

state.backend.rocksdb.ttl.compaction.filter.enabled: "true"

Tried adding this property explicity ( documentation says its enabled by default), but there is no improvement. Enabled RocksDB metrics, I can see the estimated keys metrics is returning to 0 after load runs. Is there any way to force compactions to happen periodically irrespective of the size of expired state?

I’m in a similar boat, using

ListState

(with Flink

1.17.x

) and found this post with someone in a similar situation: • https://stackoverflow.com/questions/76569551/flink-checkpoint-size-grows-continuously I’ve tried everything, from adding TTL on the ListState to try to tune rocksdb to enable compaction and/or eviction of stale state. Nothing seems to help. Anyone else experiencing this problem?

Facing the same issue, were you guys able to tackle it? @mike Trienis @abhishek awasthi

@Pranay Das yup, just switch to the

HashMapStateBackend

instead of

EmbeddedRocksDBStateBackend