Anyone noticed bad performance on the Pinot UI? Sp...
# general
j
Anyone noticed bad performance on the Pinot UI? Specifically table pages. It looks like time gets sucked up calculating table size. Is there a way to improve performance or turn off the size calcs?
b
That is true, especially for large clusters/tables. I think the reason is that some of the API endpoints were not designed with the controller UI in mind but were incidentally used. It would be great if someone could lead an effort to document these inefficiencies (maybe a github issue?) and create concrete tasks for contributors to pick up.
💯 1
t
These would be outstanding tasks for early-stage contributors to take on. Excellent idea, Caner.
Jess, all you did was ask an innocent question, and here you are getting [the suggestion of] work assigned to you. 😆
But seriously, if you could make a list and even just share it here, that would be hugely helpful.
j
Slow API endpoints?
console.png
b
My experience of UI slowness mainly comes from ‘too many requests’ rather than slow endpoints. But this is a good find as well. Worth looking into why it takes so long.
It is possible your ZK cluster is overloaded. I’d recommend looking at ZK related metrics your controller emits, such as ‘helixzkclient.readlatencygauge’
🤔 1
💥 1
j
it’s somewhat unavoidable since the controller is reaching out to all servers, and they need to collect a bunch of data from disk while also they may be performing a ton of work. and sometimes servers don’t respond, so the controllers has to wait for the defualt http timeout
now that said, once you upgrade to pinot 1.1, I tried to work around this in the UI in https://github.com/apache/pinot/pull/12210 by gradually rendering the UI everywhere I could. This /size issue was one of the top things I had in mind
🔥 2
we have clusters with hundreds of tables and millions of segments, so we see the same slowness you do. but ☝️ should at least render everything it can as soon as it can. but if you feel inclined to fix the actual APIs to be faster, that would also be great