When using Flink on k8s with the Operator (<https:...
# troubleshooting
h
When using Flink on k8s with the Operator (https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.9/) it is possible to set resource limits&requests for both the jobmanager and the taskmanager. Unfortunately, it seems the operator sets both the limit and the requests to the same value, resulting in a “guaranteed” claim on k8s. To better use the resources in our cluster I’d really like to use the burstable mode, for instance giving the jobmanager 1/10 core but allow it to burst to 2 cores during startup if there’s resources available. The same would also be useful for the taskmanagers - give them enough resources for the steady load but allow them to burst when there’s a need. Is this something we can do?
m
You can use the Flink configurations
kubernetes.jobmanager.cpu.limit-factor
and
kubernetes.taskmanager.cpu.limit-factor
. The same exists for memory as well: `kubernetes.jobmanager.memory.limit-factor`and
kubernetes.taskmanager.memory.limit-factor
. By default it is 1.0, if you set it to e.g. 2.0, the limit will be 2 times the request amount
☝️ 1
h
Interesting, will give it a try tomorrow 🙂