Hi everone, I am working with the Apache Flink Ope...
# troubleshooting
a
Hi everone, I am working with the Apache Flink Operator but I have an issue with the getParallelism method, which always override my parallelism setting in yaml configuration. Let's say I have a Flink application running with Application Mode and Standalone which has 3 operator tasks and they run in their own slot sharing groups, e.g. operator A runs in SlotSharingGroup A, operator B runs in SlotSharingGroup B,...Its parallelism is 2, so the Flink job needs at least 6 task slots. If
taskmanager.numberOfTaskSlots
is 2, the
taskManager.replicas
should be at least 3. However, with the implemenation of the getParallelism method, the parallelism.default will be *6 = `taskmanager.numberOfTaskSlots * taskManager.replicas`* , so it overrides my parallelism setting of 2. Therefore, I think the logic of the getParallelism method, does not make sense to me. Do I miss anything? Thanks
g
Bumping this up for visibility. Has anyone seen this and understood why the implementation is as such?
g
It’s impossible to know what slot sharing groups the job defines
This is a best effort estimation.
Please use the native mode instead for such setup :)
gratitude thank you 2