Hello community, am I missing something or is it not possible to get a metric with state size per state store and task manager/slot?
Currently we are using checkpoint size as proxy but it’s not ideal given there can be data skews on some partitions?
Using in memory state
d
David Anderson
10/02/2025, 5:26 PM
With the in-memory, hashmap state backend, I would expect the checkpoint sizes to be an accurate measure of state size. What makes you think it's less than ideal? The checkpoint size metric is reported on a per-subtask level, so any skew will be apparent.
l
Lukasz Krawiec
10/02/2025, 6:13 PM
Thanks for the response, that's exactly how I expected this metric to be published but I only see checkpoint size metric coming from JM as a total. I'll revisit my metric configs