Hi team, i triggered server rebalance in my pinot ...
# general
i
Hi team, i triggered server rebalance in my pinot cluster with 3 servers, but the segment reassignment shows the target server for all the segments as only one server. How can i make it assign the data to all 3 servers.
partition per topic for my kafka is 1 currently, so only one server is consuming data
m
Yes, you need to increase the number of partitions
y
Yeah 150 TB of data on a single partition is certainly not recommended. But if repartitioning upstream isn't possible, on the Pinot side you could configure instance assignment for
COMPLETED
segments separately (see https://docs.pinot.apache.org/operators/operating-pinot/instance-assignment for more details).
CONSUMING
segments will always be assigned to a single server per partition (and per replica).
👍 1
thanks 1
i
this one partition is for the testing cluster, for prod we will have more than 100 partitions.