Hello everyone! I've been using Pinot for a few mo...
# troubleshooting
e
Hello everyone! I've been using Pinot for a few months. I noticed that my realtime table segments are stored on the same server when they should be balanced on all my servers by default. (that's what i'm looking for) I have four servers with the tag "*DefaultTenant_REALTIME*" and a table with 80 segments stored on my second server. Am I missing something important ?
l
you mean that all your segments are hosted in one sever instead of being evenly distributed among the 4?
k
How many partitions in your pulsar topic?
e
Yes, all my segments are hosted on the same server. The pulsar topic is not partitioned, I kept the default settings
k
That’s the reason
e
Thanks a lot, i'll check that But I don't understand why do i have to change this parameter ?
k
Because we distribute the number of partitions evenly among the servers
If there is only 1 partitio, we can do much
You can however create use another to move the segments to other servers as they get completed
l
would you ideally recommend to have one server per partition (?)
e
if I understand correctly there can be only one server that consumes per partition and we can rebalance all the segments once they are all online We have tried this but the rebalance server action tells us that the segments are already balanced on Pinot
n
you need a special config to be able to move completed segments to other servers. By default, even the complted segments will stay on the same node as the CONSUMING one, for a given partition. We need this sticky routing for partition based routing, or upserts to work correctly. If you dont care about either of these, let me share that config.
in any case, not a good idea to go to production with just 1 partition in your topic. because even if we distribute the completed segments, the ingestion will remain on the same node
e
ok I understand better now, i will add partitioning on Pulsar and I won't need rebalance server action after that thank you all for your answers