Hi Folks, I am using prestodb, but fond an issue w...
# general
c
Hi Folks, I am using prestodb, but fond an issue when using order by , Could anyone help check?
d
Can you add some logs? That may help to find the issue. See if adding --debug to your presto CLI the query prints a stack trace
x
Can you explain this query in presto and use right arrow to see the generated Pinot query?
k
@User please try removing the semi-colon at the end of the query. I've found this to be an issue with
ORDER-BY
queries recently. Looking into it further.
c
Hi @User
presto:default> explain select * from matsmeeting order by jobprocesstime desc;
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - Output[servicetype, hostuserid, confopentimets, modifiedcount, endtimewwpts, hasrecording, confopentime, confname, haschat, h264sharing, endtimewwp, isenablewebcast, state, closerea - RemoteStreamingMerge[jobprocesstime DESC_NULLS_LAST] => [servicetype:varchar, hostuserid:varchar, confopentimets:timestamp, modifiedcount:integer, endtimewwpts:timestamp, hasrec - LocalMerge[jobprocesstime DESC_NULLS_LAST] => [servicetype:varchar, hostuserid:varchar, confopentimets:timestamp, modifiedcount:integer, endtimewwpts:timestamp, hasrecording - PartialSort[jobprocesstime DESC_NULLS_LAST] => [servicetype:varchar, hostuserid:varchar, confopentimets:timestamp, modifiedcount:integer, endtimewwpts:timestamp, hasreco - RemoteStreamingExchange[REPARTITION] => [servicetype:varchar, hostuserid:varchar, confopentimets:timestamp, modifiedcount:integer, endtimewwpts:timestamp, hasrecordi Estimates: {rows: ? (?), cpu: ?, memory: 0.00, network: ?} - TableScan[TableHandle {connectorId=‘pinot’, connectorHandle=’PinotTableHandle{connectorId=pinot, schemaName=default, tableName=matsmeeting, isQueryShort=Optional Estimates: {rows: ? (?), cpu: ?, memory: 0.00, network: 0.00} attendeenum := PinotColumnHandle{columnName=attendeeNum, dataType=integer, type=REGULAR} siteid := PinotColumnHandle{columnName=siteId, dataType=bigint, type=REGULAR} ints := PinotColumnHandle{columnName=ints, dataType=timestamp, type=REGULAR} meetingtype := PinotColumnHandle{columnName=meetingType, dataType=varchar, type=REGULAR} confopentime := PinotColumnHandle{columnName=confOpenTime, dataType=varchar, type=REGULAR} reporttime := PinotColumnHandle{columnName=reportTime, dataType=varchar, type=REGULAR} webexdeviceidtype := PinotColumnHandle{columnName=webexDeviceIdType, dataType=varchar, type=REGULAR} hostuserid := PinotColumnHandle{columnName=hostUserId, dataType=varchar, type=REGULAR} hasdocumentsharing := PinotColumnHandle{columnName=hasDocumentSharing, dataType=varchar, type=REGULAR} userid := PinotColumnHandle{columnName=userId, dataType=bigint, type=REGULAR} confid := PinotColumnHandle{columnName=confId, dataType=bigint, type=REGULAR} jobprocessts := PinotColumnHandle{columnName=jobProcessTS, dataType=timestamp, type=REGULAR} hastpuser := PinotColumnHandle{columnName=hasTPUser, dataType=varchar, type=REGULAR} ismct := PinotColumnHandle{columnName=isMCT, dataType=varchar, type=REGULAR} endtimewwpts := PinotColumnHandle{columnName=endTimeWWPTS, dataType=timestamp, type=REGULAR} hasapplicationsharing := PinotColumnHandle{columnName=hasApplicationSharing, dataType=varchar, type=REGULAR} modifiedcount := PinotColumnHandle{columnName=modifiedCount, dataType=integer, type=REGULAR} featurename := PinotColumnHandle{columnName=featureName, dataType=varchar, type=REGULAR} closereason := PinotColumnHandle{columnName=closeReason, dataType=varchar, type=REGULAR} lts := PinotColumnHandle{columnName=lts, dataType=timestamp, type=REGULAR} statewwp := PinotColumnHandle{columnName=stateWWP, dataType=integer, type=REGULAR} endtimets := PinotColumnHandle{columnName=endTimeTS, dataType=timestamp, type=REGULAR} hostnodeid := PinotColumnHandle{columnName=hostNodeId, dataType=varchar, type=REGULAR} endtimewwp := PinotColumnHandle{columnName=endTimeWWP, dataType=varchar, type=REGULAR} hasrecording := PinotColumnHandle{columnName=hasRecording, dataType=varchar, type=REGULAR} numwbxusers := PinotColumnHandle{columnName=numWbxusers, dataType=integer, type=REGULAR} jobpoolname := PinotColumnHandle{columnName=jobPoolName, dataType=varchar, type=REGULAR} clusterid := PinotColumnHandle{columnName=clusterId, dataType=varchar, type=REGULAR} duration := PinotColumnHandle{columnName=duration, dataType=integer, type=REGULAR} haschat := PinotColumnHandle{columnName=hasChat, dataType=varchar, type=REGULAR} env := PinotColumnHandle{columnName=env, dataType=varchar, type=REGULAR} cmrversioncode := PinotColumnHandle{columnName=cmrVersionCode, dataType=varchar, type=REGULAR} isenablewebcast := PinotColumnHandle{columnName=IsEnableWebcast, dataType=varchar, type=REGULAR} jobprocesstime := PinotColumnHandle{columnName=jobProcessTime, dataType=varchar, type=REGULAR} lastmodifiedtime := PinotColumnHandle{columnName=lastModifiedTime, dataType=varchar, type=REGULAR} nbrenabled := PinotColumnHandle{columnName=NBREnabled, dataType=varchar, type=REGULAR} confname := PinotColumnHandle{columnName=confName, dataType=varchar, type=REGULAR} ts := PinotColumnHandle{columnName=ts, dataType=timestamp, type=REGULAR} starttimets := PinotColumnHandle{columnName=startTimeTS, dataType=timestamp, type=REGULAR} isspacemeeting := PinotColumnHandle{columnName=isSpaceMeeting, dataType=varchar, type=REGULAR} tpserver := PinotColumnHandle{columnName=tpServer, dataType=varchar, type=REGULAR} endtime := PinotColumnHandle{columnName=endTime, dataType=varchar, type=REGULAR} sitename := PinotColumnHandle{columnName=siteName, dataType=varchar, type=REGULAR} starttime := PinotColumnHandle{columnName=startTime, dataType=varchar, type=REGULAR} state := PinotColumnHandle{columnName=state, dataType=integer, type=REGULAR} confopentimets := PinotColumnHandle{columnName=confOpenTimeTS, dataType=timestamp, type=REGULAR} haspolling := PinotColumnHandle{columnName=hasPolling, dataType=varchar, type=REGULAR} componentaddress := PinotColumnHandle{columnName=componentAddress, dataType=varchar, type=REGULAR} meetingtypecode := PinotColumnHandle{columnName=meetingTypeCode, dataType=varchar, type=REGULAR} hasfilesharing := PinotColumnHandle{columnName=hasFileSharing, dataType=varchar, type=REGULAR} isenableevent := PinotColumnHandle{columnName=IsEnableEvent, dataType=varchar, type=REGULAR} numtpusers := PinotColumnHandle{columnName=numTpUsers, dataType=integer, type=REGULAR} h264sharing := PinotColumnHandle{columnName=H264Sharing, dataType=varchar, type=REGULAR} servicetype := PinotColumnHandle{columnName=serviceType, dataType=varchar, type=REGULAR} mconline := PinotColumnHandle{columnName=MCONLINE, dataType=varchar, type=REGULAR} mtgserver := PinotColumnHandle{columnName=mtgServer, dataType=varchar, type=REGULAR} meetingkey := PinotColumnHandle{columnName=meetingKey, dataType=varchar, type=REGULAR} supportwebexcloudproximity := PinotColumnHandle{columnName=supportWebexCloudProximity, dataType=varchar, type=REGULAR} closereasoncode := PinotColumnHandle{columnName=closeReasonCode, dataType=varchar, type=REGULAR} clientipaddr := PinotColumnHandle{columnName=clientIpAddr, dataType=varchar, type=REGULAR}
Hi @User here is debug info
presto:default> select * from matsmeeting order by jobprocesstime desc;
Query 20211013_011448_00018_92gth, FAILED, 1 node http://localhost:8080/ui/query.html?20211013_011448_00018_92gth Splits: 118 total, 0 done (0.00%) CPU Time: 0.0s total, 0 rows/s, 0B/s, 0% active Per Node: 0.0 parallelism, 0 rows/s, 0B/s Parallelism: 0.0 Peak Memory: 0B 0:00 [0 rows, 0B] [0 rows/s, 0B/s] Query 20211013_011448_00018_92gth failed: null value in entry: Server_sj1-pinot-server-34_8098=null java.lang.NullPointerException: null value in entry: Server_sj1-pinot-server-34_8098=null at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:32) at com.google.common.collect.SingletonImmutableBiMap.<init>(SingletonImmutableBiMap.java:42) at com.google.common.collect.ImmutableBiMap.of(ImmutableBiMap.java:72) at com.google.common.collect.ImmutableMap.of(ImmutableMap.java:124) at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:458) at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:437) at com.facebook.presto.pinot.PinotSegmentPageSource.queryPinot(PinotSegmentPageSource.java:242) at com.facebook.presto.pinot.PinotSegmentPageSource.fetchPinotData(PinotSegmentPageSource.java:214) at com.facebook.presto.pinot.PinotSegmentPageSource.getNextPage(PinotSegmentPageSource.java:161) at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:252) at com.facebook.presto.operator.Driver.processInternal(Driver.java:418) at com.facebook.presto.operator.Driver.lambda$processFor$9(Driver.java:301) at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:722) at com.facebook.presto.operator.Driver.processFor(Driver.java:294) at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077) at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162) at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:599) at com.facebook.presto.$gen.Presto_0_261_SNAPSHOT_4175e47____20211008_012118_1.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
@User I found if i remove semi-colon, i can’t run commend in cli
message has been deleted
k
Ah okay. There is a bug in 0.9.0 where a semi-colon at the end of a query in the browser does result in an error. But this looks very much related to Presto.
c
Hi @User It’s strange that only failed when query contain order-by?
k
The exception doesn’t seem to be happening in the Pinot runtime. This looks like a bug in Presto. I’m not very familiar with Presto’s code so I think someone else might be better able to help you here. I would, however, try running your query in the Pinot query console and see if it returns any results.
c
Thx @User Do we have presto coder in this room? 😀
k
Yes, I think @User might be able to help if he has bandwidth. If not, we will find someone to help.
c
Hi @User Could you please help look at this issue, thx 😀
d
I've just got this error as well but my query doesn't have a group/order by, is just a simple
select sum(1) from table
x
semi colon is required from presto side to end a query
for your query explain, can you use right arrow -> to look at more plans from right side?
one screen width is not enough to show the full plan
there should be something more after
TableScan[TableHandle {connectorId='pinot', connectorHandle='PinotTableHandle{connectorId=pinot, schemaName=default, tableName=matsmeeting, isQueryShort=Optional
my feeling is that this query is not a short query, so presto is asking all data from pinot then sort by itself
so the order by is not pushed down to pinot
which should be something that can be added into Presto-Pinot connector
c
Yes @User Seems presto issue and jdbc is fine for us
Sorry I don’t know how to use -> this,
presto:default> explain select * from matsmeeting order by jobprocesstime desc ->;
Query 20211018_061237_00003_92gth failed: line 164 mismatched input ‘->‘. Expecting: ‘,’, ‘LIMIT’, ‘NULLS’, ‘OFFSET’, <EOF> explain select * from matsmeeting order by jobprocesstime desc -> presto:default> explain -> select * from matsmeeting order by jobprocesstime desc ; Query 20211018_061249_00004_92gth failed: line 19 mismatched input ‘->‘. Expecting: ‘(’, ‘ALTER’, ‘ANALYZE’, ‘CALL’, ‘COMMIT’, ‘CREATE’, ‘DEALLOCATE’, ‘DELETE’, ‘DESC’, ‘DESCRIBE’, ‘DROP’, ‘EXECUTE’, ‘EXPLAIN’, ‘GRANT’, ‘INSERT’, ‘PREPARE’, ‘REFRESH’, ‘RESET’, ‘REVOKE’, ‘ROLLBACK’, ‘SET’, ‘SHOW’, ‘START’, ‘USE’, ‘VERBOSE’, <query> explain -> select * from matsmeeting order by jobprocesstime desc presto:default> explain > select * from matsmeeting order by jobprocesstime desc; Query 20211018_061322_00005_92gth failed: line 19 mismatched input ‘>’. Expecting: ‘(’, ‘ALTER’, ‘ANALYZE’, ‘CALL’, ‘COMMIT’, ‘CREATE’, ‘DEALLOCATE’, ‘DELETE’, ‘DESC’, ‘DESCRIBE’, ‘DROP’, ‘EXECUTE’, ‘EXPLAIN’, ‘GRANT’, ‘INSERT’, ‘PREPARE’, ‘REFRESH’, ‘RESET’, ‘REVOKE’, ‘ROLLBACK’, ‘SET’, ‘SHOW’, ‘START’, ‘USE’, ‘VERBOSE’, <query> explain > select * from matsmeeting order by jobprocesstime desc presto:default> explain select * from matsmeeting order by jobprocesstime desc >; Query 20211018_061329_00006_92gth failed: line 165 mismatched input ‘>’. Expecting: ‘,’, ‘LIMIT’, ‘NULLS’, ‘OFFSET’, <EOF> explain select * from matsmeeting order by jobprocesstime desc > presto:default> explain select * from matsmeeting order by jobprocesstime desc ->; Query 20211018_061332_00007_92gth failed: line 164 mismatched input ‘->‘. Expecting: ‘,’, ‘LIMIT’, ‘NULLS’, ‘OFFSET’, <EOF> explain select * from matsmeeting order by jobprocesstime desc ->
x
Oh, it’s just the right arrow(one of the 4 directions ) on your keyboard to see more content