Hi, I am facing an issue with offline servers on r...
# pinot-perf-tuning
s
Hi, I am facing an issue with offline servers on running heavy queries via presto. I have 6 offline servers each with 32G RAM and I have configured my tables to have 2 RGs with 3 servers each. We have major chunk of our data (some tables of size 29G) in offline servers and that would keep increasing with time. When I run queries, most of the times the server goes down due to OOM or the query gets aborted due to some exception. Can I get some insights on the configuration for heap/non-heap allocation?
m
what’s your xmx? Also what query is presto sending to pinot (you can find via explain plan)
s
Have been trying different values for that. Xmx right now is 8G and directMemory is 20G. This isn't working and giving grpc status unknown exception. Before, I had xmx as 15G and directMemory as 13G which wouldn't release memory even after the query was complete. Also, queries aren't sent to pinot brokers as checked using explain query.
@Mayank could the grpc server involved in data transfer cause memory leakage?
m
It seems odd. memory should be freed. @Xiang Fu any thoughts?