Seeing some query failures and seeing the following errors in Pinot server intermittently

2021/07/16 18:00:42.408 INFO [QueryScheduler] [pqr-1] Processed requestId=55381,table=domainEventView_REALTIME,segments(queried/processed/matched/consuming)=20/14/2/3,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=1,resSerMs=0,totalTimeMs=2,minConsumingFreshnessMs=1626458256041,broker=Broker_pinot-broker-1.pinot-broker.traceable.svc.cluster.local_8099,numDocsScanned=209,scanInFilter=940,scanPostFilter=426,sched=fcfs
2021/07/16 18:00:42.407 ERROR [ServerQueryExecutorV1Impl] [pqr-0] Exception processing requestId 55381
java.lang.RuntimeException: Caught exception while running CombinePlanNode.
java.lang.RuntimeException: Caught exception while running CombinePlanNode.
	at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
	at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
	at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
	at java.util.concurrent.FutureTask.report(Unknown Source) ~[?:?]
	at java.util.concurrent.FutureTask.get(Unknown Source) ~[?:?]
	at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:147) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
	... 15 more
Caused by: java.lang.NullPointerException

Looks like the last bit of the stack trace might be missing (which would show where the NPE is occurring)

@Ken Krugler: This is the full log

It this the first time this is logged? The actual exception which causes the filure is the NPE, but there is no stack trace for that

Try adding

-XX:-OmitStackTraceInFastThrow

to jvm args and retry.