Hi folks :wave: When it comes to Partition discove...
# troubleshooting
d
Hi folks πŸ‘‹ When it comes to Partition discovery in Flink, the KafkaSourceEnumerator is able to find new partitions and assign them to task managers. But is there a way to have Flink rebalance partition assignment if we later increase the parallelism? For example if I have 6 kafka partitions assigned to 5 task managers, one of the TMs will have 2 partitions. If I redeploy my app from a snapshot with an increased parallelism of 6, the partitions are not rebalanced - so one of the TMs has no partitions assigned and halts watermarks. While I could set the idleness to prevent watermark halting, that's not really the problem i'm chasing. I instead want to have the partitions rebalanced, so each TM is getting 1 partition. Is there a way to achieve this? Is it not considered a safe operation for processing guarantees?
❓ 1