Apache Flink

The Async duration in my operator checkpointing are higher even though Flink has room in managed memory, disk iops and disk bandwidth.
What other constraints should I be looking at? I'm using aligned checkpoints

&gt; *Async Duration*: The duration of the asynchronous part of the checkpoint. This includes time it took to write the checkpoint on to the selected filesystem. For unaligned checkpoints this also includes also the time the subtask had to wait for last of the checkpoint barriers to arrive (alignment duration) and the time it took to persist the in-flight data

IMO the data skew is a pretty big contributor.

One observation I made is that CPU Usage % spiked during this time interval.

I'm using Flink Kubernetes operator the CPU allocation to all TM pods is uniform and static. I wonder if there is a way to efficiently manage CPUs by allocation more to the TM pods that require it

```  taskmanager.memory.process.size: "70304m"
  jobmanager.memory.process.size: "70304m"
  kubernetes.jobmanager.cpu.limit-factor: "2"
  kubernetes.taskmanager.cpu.limit-factor: "2"

jobManager:
  resource:
    memory: "70304m"
    cpu: 8
taskManager:
  resource:
    memory: "70304m"
    cpu: 8```