We are running Flink 1.17 and have below configura...
# troubleshooting
p
We are running Flink 1.17 and have below configuration
Copy code
taskmanager.memory.process.size: 1536m
  taskManager:
    resource:
      memory: 2048m
We expected the taskmanager pod to be created with 2048MB memory and only 1536MB to be allocated to “Total Process Memory”. But the job started with 2048m “Total Process Memory”. It seems
taskmanager.memory.process.size
setting was ignored. We would like to start have Flink’s Total Process Memory lower than pod’s memory limits. Can someone please help us with it ?
g
The tm.resource.memory simply sets the process size so it is naturally overriten. That is by default both the memory request and limit in the same number
if you want to set a higher limit you can use:
Copy code
kubernetes.taskmanager.memory.limit-factor: 1.25
this would set a 25% higher limit for the TM pod
so around 2500 in this case
hope that helps
p
Thanks for the suggestion. Also do we have any way to set the “total process memory” set by flink to be lower than pod’s requested memory ?
g
I don’t think so.
p
This is how memory utilisation of task manager pod looks like, it stays above the requested memory for all the time. So we were wondering if we can configure taskmanager memory to stay under requests for normal operations