https://pinot.apache.org/ logo
Join Slack
Powered by
Just reported <https://github.com/apache/incubator...
# general
c
Just reported https://github.com/apache/incubator-pinot/issues/6253, 
NullPointerException
is thrown when query with aggregation on top of groovy functions. Should we expect aggregation to work on top of groovy as it’s treated as transform function?
From the code, the NPE is thrown at,
Copy code
operatorLatch.countDown();
Which doesn’t make a lot sense to me as it should never be NULL…
c
might be better to continue in troubleshooting channel ? @Neha Pawar @Jackie FYI
n
this works for me:
Copy code
select DestState, avg(groovy('{"returnType":"DOUBLE","isSingleValue":true}', 'arg0 > arg1 ? arg0.toDouble() : arg1.toDouble()', ActualElapsedTime, AirTime)) as theAvg from airlineStats where ActualElapsedTime>0 and AirTime>0 group by DestState limit 10
im on master though
nothing wrong with the query should’ve worked. let me investigate more
for 0.5.0 looks like the line throwing the NPE is 
_indexedTable.finish(false);
any exceptions before this one?
c
I don’t see other exceptions from the query response.
n
how about in the server logs?
c
let me rerun the query and see if I can find relevant log on brokers.
n
please check server also. this exception is from the server code
c
server you mean the nodes, right?
n
i mean pinot-server logs. Check that along with pinot-broker
c
I found below error on server,
Copy code
java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Double
	at org.apache.pinot.core.operator.transform.function.GroovyTransformFunction.transformToDoubleValuesSV(GroovyTransformFunction.java:280) 
	at org.apache.pinot.core.operator.docvalsets.TransformBlockValSet.getDoubleValuesSV(TransformBlockValSet.java:85) 
	at org.apache.pinot.core.query.aggregation.function.AvgAggregationFunction.aggregateGroupBySV(AvgAggregationFunction.java:103) 
	at org.apache.pinot.core.query.aggregation.groupby.DefaultGroupByExecutor.aggregate(DefaultGroupByExecutor.java:143) 
	at org.apache.pinot.core.query.aggregation.groupby.DefaultGroupByExecutor.process(DefaultGroupByExecutor.java:130) 
	at org.apache.pinot.core.operator.query.AggregationGroupByOrderByOperator.getNextBlock(AggregationGroupByOrderByOperator.java:106) 
	at org.apache.pinot.core.operator.query.AggregationGroupByOrderByOperator.getNextBlock(AggregationGroupByOrderByOperator.java:38) 
	at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) 
	at org.apache.pinot.core.operator.combine.GroupByOrderByCombineOperator$1.runJob(GroupByOrderByCombineOperator.java:137) 
	at org.apache.pinot.core.util.trace.TraceRunnable.run(TraceRunnable.java:40) 
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_265]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_265]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_265]
	at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) 
	at shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) 
	at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_265]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_265]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
Why is it casting things to double?
n
2 reasons: 1. In your groovy query, i see return type double. So the final result returned by your script should be double. For instance, in the sample query I shared, my arg0 and arg1 were INT, so I used 
toDouble()
on them 2. AVG, SUM functions expect double (i think)
it could be failing for you in one of these 2 palces
c
AH
let me try converting everything to double first. 🙂
hmm… now i’m getting below error with 
toDouble()
i think. Let me play around with it a bit. Thanks for the help!!
Copy code
java.lang.ClassCastException: java.math.BigDecimal cannot be cast to java.lang.Double
	at org.apache.pinot.core.operator.transform.function.GroovyTransformFunction.transformToDoubleValuesSV(GroovyTransformFunction.java:280)
	at org.apache.pinot.core.operator.docvalsets.TransformBlockValSet.getDoubleValuesSV(TransformBlockValSet.java:85)
	at org.apache.pinot.core.query.aggregation.function.AvgAggregationFunction.aggregateGroupBySV(AvgAggregationFunction.java:103)
I eventually get it to work. So to sum up, 1. Avg/Sum/PercentileTDigest## would all require 
double
so need to convert the all possible return values to 
double
; 2. I have a number literal, 
0.0
, which I thought would make it double but actually is bigDecimal. So instead I used 
0d
and second exception is gone.
n
👌
c
Much appreciated, @Neha Pawar!
🙂 1
🙏
Open in Slack
PreviousNext
Powered by