Kishore G
Jonathan Meyer
07/13/2021, 3:25 PMSaurabh Dwivedy
07/14/2021, 1:55 PMKamal Chavda
07/14/2021, 2:38 PMLuiz Gabriel Lima Pinheiro
07/14/2021, 4:09 PMBruce Ritchie
07/14/2021, 8:48 PMDeepak Mishra
07/15/2021, 11:27 AMAzri Jamil
07/15/2021, 12:55 PMWARN [ZkBaseDataAccessor] [ZkClient-EventThread-17-pinot-zookeeper:2181] Fail to read record for paths: {/mdm-analytic/INSTANCES/Server_pinot-server-0.pinot-server-headless.default.svc.cluster.local_8098/MESSAGES/219cd205-8c0c-455b-b509-d26b525dfa7c=-101, /mdm-analytic/INSTANCES/Server_pinot-server-0.pinot-server-headless.default.svc.cluster.local_8098/MESSAGES/269cb90d-0d58-473c-b8ca-7ef892d69da5=-101, /mdm-analytic/INSTANCES/Server_pinot-server-0.pinot-server-headless.default.svc.cluster.local_8098/MESSAGES/0142c30b-740d-4f80-ab61-87110deb8783=-101}
Laxman Ch
07/16/2021, 6:11 PM2021/07/16 18:00:42.408 INFO [QueryScheduler] [pqr-1] Processed requestId=55381,table=domainEventView_REALTIME,segments(queried/processed/matched/consuming)=20/14/2/3,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=1,resSerMs=0,totalTimeMs=2,minConsumingFreshnessMs=1626458256041,broker=Broker_pinot-broker-1.pinot-broker.traceable.svc.cluster.local_8099,numDocsScanned=209,scanInFilter=940,scanPostFilter=426,sched=fcfs
2021/07/16 18:00:42.407 ERROR [ServerQueryExecutorV1Impl] [pqr-0] Exception processing requestId 55381
java.lang.RuntimeException: Caught exception while running CombinePlanNode.
java.lang.RuntimeException: Caught exception while running CombinePlanNode.
at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.InstanceResponsePlanNode.run(InstanceResponsePlanNode.java:33) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:45) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:294) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:215) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.executor.QueryExecutor.processQuery(QueryExecutor.java:60) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:157) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:141) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(Unknown Source) ~[?:?]
at java.util.concurrent.FutureTask.get(Unknown Source) ~[?:?]
at org.apache.pinot.core.plan.CombinePlanNode.run(CombinePlanNode.java:147) ~[pinot-all-hypertrace-0.7.1-5-shaded.jar:0.7.1-24ebc6d4cd25b1b21225f48c4e7438919246ffe3]
... 15 more
Caused by: java.lang.NullPointerException
Ken Krugler
07/16/2021, 9:59 PMselect sum(metric), key, min(date) as firstSeen, max(date) as lastSeen from table where date >= <lowDate> AND date <= <highDate> group by key order by firstSeen desc limit 1
. I get a single row as expected, but with a firstSeen
and lastSeen
both equal to <highDate>
. I was expecting the firstSeen
result to be equal to <lowDate>
. If I then run the exact same query, but add in AND key = '<key value from the previous result>'
, I get the a single row with the requested key value, but now the firstSeen
result is equal to <lowDate>
(as expected), and sum(metric)
is larger (also as expected). Any ideas what is going on?Jonathan Meyer
07/19/2021, 5:00 PMCharles
07/20/2021, 12:26 AMYash Agarwal
07/21/2021, 10:03 AMselect distinct DATETIMECONVERT(transaction_date, '1:DAYS:EPOCH', '1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd', '1:DAYS') from transactions limit 1000 -- 80+ seconds
select distinct transaction_date from transactions limit 1000 -- 3.5 seconds
Can you help with the how to optimize the same. In the meanwhile we have added another column in the yyyy-MM-dd
format to support the same.Jonathan Meyer
07/21/2021, 9:27 PMSadim Nadeem
07/22/2021, 4:51 PMRK
07/23/2021, 6:25 AMDeepak Mishra
07/23/2021, 12:03 PMSyed Akram
07/23/2021, 3:51 PMDeepak Mishra
07/25/2021, 3:42 AMCarlos Domínguez
07/26/2021, 2:00 PMII
07/26/2021, 8:46 PMsuraj kamath
07/28/2021, 2:20 PMbin/pinot-admin.sh UploadSegment -controllerHost localhost -controllerPort 9000 -segmentDir ${PINOT_ROOT}/pinot/server/data/index/transcript_REALTIME/transcript__0__4__20210720T0957Z -tableName transcript_OFFLINE
Executing command: UploadSegment -controllerProtocol http -controllerHost localhost -controllerPort 9000 -segmentDir ${PINOT_ROOT}/pinot/server/data/index/transcript_REALTIME/transcript__0__4__20210720T0957Z
Compressing segment: v3
Uploading segment tar file: /var/folders/fp/dgy2h98d2639nyd4q_v2bwt40000gn/T/segmentUploader4462261101415652801.tmp/v3.tar.gz
Sending request: <http://localhost:9000/v2/segments?tableName=transcript_OFFLINE> to controller: 192.168.0.105, version: Unknown
With this I am able to see the segment under the OFFLINE table, and query the table for data as well. However the segment Meta Data seems to be improper/corrupt and I see the error(refer attached pic)
Am I missing something here ?
Please let me know
Thanks
PS:
I have a hybrid table setup, and am exploring uploading of segments directly from realtime to offline tables as part of a POC.Deepak Mishra
07/29/2021, 5:12 AMbeerus
07/29/2021, 12:30 PMKamal Chavda
07/29/2021, 8:33 PM{"name":"value"}
I'm trying to get the "value" by passing in the "name". I tried the JSON transformations but keep getting errors. I didn't see any string-to-json transformations on the 'supported transformations' page. Any other documentation with relevant information or suggestions?Kenneth Koo
07/30/2021, 12:40 AMDunith Dhanushka
select * from steps where loggedAt > now() - cast(60*60*24*1000 as long)
I have records that falls into this bucket. But this query returns nothing.
Where did it go wrong?Deepak Mishra
07/30/2021, 4:25 AMkhush
07/30/2021, 10:41 AMMap
07/30/2021, 2:31 PMtrino:default> select count(*) from table1 group by MSGTIME limit 10;
Query 20210730_141801_00012_uani3, FAILED, 1 node
Splits: 51 total, 0 done (0.00%)
0.16 [0 rows, 0B] [0 rows/s, 0B/s]
Query 20210730_141801_00012_uani3 failed: Segment query returned '50001' rows per split, maximum allowed is '50000' rows. with query "SELECT MSGTIME FROM table1_REALTIME LIMIT 50001"