Hi Team, Recently I was trying to enable OOM (<htt...
# troubleshooting
m
Hi Team, Recently I was trying to enable OOM (https://docs.pinot.apache.org/operators/operating-pinot/oom-protection-using-automatic-query-killing). I have added below configurations in both broker and server config files.
Copy code
pinot.broker.instance.enableThreadCpuTimeMeasurement=true
pinot.broker.instance.enableThreadAllocatedBytesMeasurement=true
pinot.server.instance.enableThreadAllocatedBytesMeasurement=true
pinot.server.instance.enableThreadCpuTimeMeasurement=true
pinot.query.scheduler.accounting.enable.thread.memory.sampling=true
pinot.query.scheduler.accounting.enable.thread.cpu.sampling=true


pinot.query.scheduler.accounting.oom.enable.killing.query=true
pinot.query.scheduler.accounting.query.killed.metric.enabled=true

pinot.query.scheduler.accounting.oom.critical.heap.usage.ratio=0.3
pinot.query.scheduler.accounting.oom.panic.heap.usage.ratio=0.3
<http://pinot.query.scheduler.accounting.sleep.ms|pinot.query.scheduler.accounting.sleep.ms>=30
pinot.query.scheduler.accounting.oom.alarming.usage.ratio=0.3
pinot.query.scheduler.accounting.sleep.time.denominator=3
pinot.query.scheduler.accounting.min.memory.footprint.to.kill.ratio=0.01

pinot.query.scheduler.accounting.factory.name=org.apache.pinot.core.accounting.PerQueryCPUMemAccountantFactory
pinot.query.scheduler.accounting.cpu.time.based.killing.enabled=true
pinot.query.scheduler.accounting.publishing.jvm.heap.usage=true
<http://pinot.query.scheduler.accounting.cpu.time.based.killing.threshold.ms|pinot.query.scheduler.accounting.cpu.time.based.killing.threshold.ms>=1000
I have run some heavy queries to test the OOM killing feature, but I don't see any killed queries in the broker/server metrics.
Copy code
SELECT accountId,countryCode,direction,day,hour,msgType,currency,topic,finalStatus,year,month,
  SUM(CASE WHEN finalStatus = 'Failed' THEN 1 ELSE 0 END) AS failed_count,
  SUM(CASE WHEN finalStatus = 'Delivered' THEN 1 ELSE 0 END) AS success_count,
  COUNT(*) AS total_records,
  COUNT(DISTINCT udrId) AS unique_udrs,
  SUM(price) AS total_revenue,
  AVG(price) AS avg_price,
  MAX(price) AS max_price,
  MIN(price) AS min_price,
  SUM(CASE WHEN errorCode > 0 THEN 1 ELSE 0 END) AS error_count,
  SUM(price * (CASE WHEN direction = 'Unknown' THEN 1 ELSE -1 END)) AS net_revenue
FROM
  dummy_table
GROUP BY
  accountId,countryCode,direction,msgType,currency,topic,finalStatus,year,month,day,hour
ORDER BY
  total_revenue DESC,
  avg_price DESC
LIMIT 1000000
Whenever I run this query, the server goes down, but no queries are terminated automatically. Can you please help me to understand if I am missing any configurations or steps to enable this feature? I did test on
apachepinot/pinot:1.5.0-SNAPSHOT-9d32f376d8-20251016, size of Heap -Xms2G -Xmx2G for server and broker.