Greetings everyone, hope you are doing well. I am experimenting with Flink and its checkpointing mechanism using HDFS to store the checkpoints. Additionally, I am using Incremental RocksDB as state backend. I am conducting some experiments in a 5-server cluster to evaluate the performance impact of the locality of TaskManagers and DataNodes during recovery. One set of the experiments involve Local recovery (during recovery the TaskManager will find in the same machine a DataNode to fetch its state) and the other Remote Recovery (TaskManagers and DataNodes are in different machine and state will be recovered from network). The issue I am having is that the performance gain in the local recovery case is much less than anticipated and I am trying to understand the reason. The application I am running is for a fixed input, resulting in approx. 11GB required state for recovery from HDFS(This is the total state size recovered from all HDFS files obtained from logs of the recovered TM). My network is 10 Gbit and I was expecting a time difference of more than 11 sec in the remote case since all the state is transferred through the network. However, the performance difference is a little more than 2 sec. This led me to believe that during local recovery there is a bottleneck that slows recovery down. Do you know what might be causing this? Finally, I am storing both TaskManager data and DataNode Data in an SSD and the taskmanager.memory.flink.size is set to 20GB. Sorry for the lengthy question, any tips are appreciated!