Apache Pinot

What’s the recommended memory setup for broker? If a node has 16GB memory, how much memory request of the pod (assume single pod on the node), heap and off heap I should configure?

Context: we had `Xms2G -Xmx12G`  and `memory.request: 1.25Gi` configured for the broker, and sometimes saw the broker auto-restarted due to OOM. So we increased the memory.request to 12Gi, the OOM issue is gone but seeing the query latency doubled during normal hours and increased 10x during peak time.

We just noticed the slowest phase is the scatter and gather, which took 80% of time

How much data have you loaded per node? What’s the latency and qps requirements 

&gt; How much data have you loaded per node?
Do you mean server? The server node have 64GB memory, 32GB heap, and avg 16GB heap used. The latency requirement is sub-10 ms. QPS is not high right now, under 10/s.

We bumped from `Xms2G` to `Xms6G` but things got worse…

Use xms=Xmx to 16GB. Optimize indexes and use local SSD