https://pinot.apache.org/ logo
Join Slack
Powered by
# troubleshooting
  • d
    Hello everybody! I wrote a custom aggregator that is working good with Pinot, the name is drawdown and i can use it with queries like: - 
    SELECT drawdown(id, amount) as dd FROM table
    - 
    SELECT drawdown(id, amount) as dd FROM table GROUP BY strategy_id
    but i get an error running 
    SELECT drawdown(id, amount) as dd FROM table GROUP BY strategy_id ORDER BY dd
    so ordering with the value of my aggregator does not work, the error i get is:
    Copy code
    "message": "QueryExecutionError:\njava.lang.NullPointerException\n\tat org.apache.pinot.core.operator.CombineGroupByOrderByOperator.getNextBlock(CombineGroupByOrderByOperator.java:208)\n\tat org.apache.pinot.core.operator.CombineGroupByOrderByOperator.getNextBlock(CombineGroupByOrderByOperator.java:60)\n\tat org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37)\n\tat org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26)\n\tat org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)\n\tat org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48)\n\tat org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:220)\n\tat org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:152)\n\tat org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:136)\n\tat java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)"
    NullPointerException here: https://github.com/apache/incubator-pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/operator/CombineGroupByOrderByOperator.java#L208
  • d
    Does anyone have any ideas about that problem?
  • k
    are you using pql or sql end point
  • d
    Hmm the default one on UI
  • d
    @Kishore G /sql endpoint
  • k
    ok
  • k
    not sure why its throwing NPE
  • k
    its probably because of the aggregation function
  • d
    @Kishore G i suppose because my aggregator has two columns drawdown(id, amount)
  • d
    with AVG() i have no problem
  • k
    that should not matter
  • d
    @Kishore G in CombineGroupByOrderByOperator.java i only changed the timeout value (to debug it easily)
  • s
    @Damiano did you put a breakpoint inside indexedTable.finish()?
  • d
    @Sidd _indexedTable.finish(false); here?
  • d
    yes
  • s
    I mean inside the function to step through it and find the source of NPE?
  • r
    what is sysdate equivalent in pinot query?
  • r
    actually i trying to compare fields with current date / time in superset
  • m
    @Damiano I see you are using ‘AS’. I think it is not supported (certainly not in order by).
  • m
    Try ‘order by drawdown(...)’
  • s
    AS is supported in ORDER BY
  • m
    Ah, I misssed that
  • d
    Problem solved! Thanks @Sidd! The error was a wrong column name inside my aggregation function, inside getResultColumnName i had drawdown(id) instead of drawdown(id,amount) updating the signature fixed the problem an everything worked as expected.
    👍 2
  • d
    is that possible somehow?
  • m
    @Shounak Kulkarni can you paste the broker log for that query? It should show time spent where
  • k
    lets start with the json response as well
  • s
    hey @Mayank and @Kishore G, saw the logs right now when u asked for them, till now we were monitoring only prometheus metrics, so logs show server were responsible for more query time. We'll see whats wrong at server side and then come back on this.... sorry for the trouble!
  • k
    do you have the numbers?
  • s
    broker sample broker log:
    Copy code
    2020/05/11 16:44:28.807 INFO [BaseBrokerRequestHandler] [jersey-server-managed-async-executor-3714] RequestId:2071317, table:usermap_REALTIME, timeMs:185, docs:165/2090142, entries:2/330, segments(queried/processed/matched/consuming):4/4/3/1, consumingFreshnessTimeMs:1589207755678, servers:4/4, groupLimitReached:false, exceptions:0, serverStats:(Server=SubmitDelayMs,ResponseDelayMs,ResponseSize,DeserializationTimeMs);pinot-server-0_R=0,96,1110,0;pinot-server-2_R=0,86,1303,0;pinot-server-3_R=0,58,513,0;pinot-server-1_R=0,186,1367,0, query:select MAX(timeSinceEpoch), cCode FROM usermap where userId = '379560001' and activityType = 'BMO' and timeSinceEpoch >= 1589177601620988 group by cCode top 10
    data on prometheus: avg query time on broker: around 240ms
  • m
    timeMs:185
1...100101102...166Latest