@Singh I have a follow up question, maybe you can quickly point me to the materials on how to clarify it.
So I'm confused about how Flink distributes the operators in the complex pipelines, and correspondingly how it choose then ultimate number of task managers.
Assume we have a simple graph
A -> B -> C -> D, taskSlots = 2, and parallelism = 100.
In this case it simply divides 100 / 2 and gets 50, each task manager running 2 instances of each operator.
But what happens when operators have heterogeneous parallelism? Say
A (20) -> B (50) -> C (200) -> D (30)
I'm confused in this case what exactly "task slots" represent actually, because in the former case it's simply the number or parallel instances of each operator per task manager. And here it's the same? So does it mean that the number of task managers would be MAX(operator parallelism) / taskSlots? So, in this case, 200 / 2 = 100?
Sorry if it's something obvious / explained in the docs. I didn't find it and it confuses me quite a bit.
Thanks!