https://flink.apache.org/ logo
Join Slack
Powered by
Is the flink operator able to start more pods base...
# troubleshooting
c
Is the flink operator able to start more pods based on load like hpa? I'm reading this (https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.6/docs/custom-resource/autoscaler/#configuration-guide) but it looks like it's only about changing operators parallelism. Is it possible to add hpa to a job managed by the operator?
d
With in-place scaling (starting from Flink 1.18), new TM will be created on-the-fly in the running cluster and the job will be redeployed. Else, the JM and TMs will be terminated and re-deployed. The amount of TM deployed will depend on the new parallelism.
In both cases, the needed number of TMs will be automatically deployed.
c
Thank you, I'll look into it
I'm playing with the `taskmanager.numberOfTaskSlots`and 
job.parallelism
. My cluster is always spinning up only 1 task manager, no matter which combination do I make. Do you know how can I make it scale not at run time but at least on boot. For example I think that 
taskmanager.numberOfTaskSlots: 1
and 
job.parallelism: 4
should create 4 TM but I only get one. Do you know what I might be missing?
Is it possible that the 
job.parallelism
is being ignored by some config on the operator? I really can't start more than 1 TM.
Just in case someone finds this in the future, I was setting the 
job.parallelism
as 
8
and 
taskmanager.numberOfTaskSlots
as 
1
and only seeing one task manager because the job code had the parallelism hardcoded to 
1
in java. So no matter which combination I tried it would always default to 
1
task manager.
Open in Slack
PreviousNext
Powered by