org.jkiss.dbeaver.model.sql.DBSQLException: SQL Er...
# troubleshooting
c
org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [65536]: Query failed (#20210720_001825_00003_844up): null value in entry: Server_sj1-pinot-server-25_8098=null at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:513) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$0(SQLQueryJob.java:444) at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:431) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:816) at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3435) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:118) at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:116) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:4686) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63) Caused by: java.sql.SQLException: Query failed (#20210720_001825_00003_844up): null value in entry: Server_sj1-pinot-server-25_8098=null at com.facebook.presto.jdbc.PrestoResultSet.resultsException(PrestoResultSet.java:1841) at com.facebook.presto.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1751) at com.facebook.presto.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:121) at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:272) at com.facebook.presto.jdbc.PrestoStatement.execute(PrestoStatement.java:230) at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330) at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:130) ... 12 more Caused by: java.lang.NullPointerException: null value in entry: Server_sj1-pinot-server-25_8098=null at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:32) at com.google.common.collect.SingletonImmutableBiMap.<init>(SingletonImmutableBiMap.java:42) at com.google.common.collect.ImmutableBiMap.of(ImmutableBiMap.java:71) at com.google.common.collect.ImmutableMap.of(ImmutableMap.java:124) at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:459) at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:438)
m
could you run
explain
to show the Pinot query?
c
Fragment 0 [SINGLE]
Output layout: [studentid, firstname, lastname, score, timestampinepoch, gender, subject] Output partitioning: SINGLE [] Stage Execution Strategy: UNGROUPED_EXECUTION Output[studentid, firstname, lastname, score, timestampinepoch, gender, subject] │ Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] │ Estimates: {rows: 10 (2.33kB), cpu: ?, memory: 0B, network: 2.33kB} └─ Limit[10] │ Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] │ Estimates: {rows: 10 (2.33kB), cpu: ?, memory: 0B, network: 2.33kB} └─ LocalExchange[SINGLE] () │ Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] │ Estimates: {rows: 10 (2.33kB), cpu: ?, memory: 0B, network: 2.33kB} └─ RemoteSource[1] Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] Fragment 1 [SOURCE] Output layout: [studentid, firstname, lastname, score, timestampinepoch, gender, subject] Output partitioning: SINGLE [] Stage Execution Strategy: UNGROUPED_EXECUTION LimitPartial[10] │ Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] │ Estimates: {rows: 10 (2.33kB), cpu: ?, memory: 0B, network: 0B} └─ TableScan[sj1_pinot:PinotTableHandle{schemaName=default, tableName=transcript, constraint=TupleDomain{ALL}, limit=OptionalLong[10], query=Optional.empty}, grouped = false] Layout: [studentid:integer, firstname:varchar, lastname:varchar, score:real, timestampinepoch:bigint, gender:varchar, subject:varchar] Estimates: {rows: ? (?), cpu: ?, memory: 0B, network: 0B} studentid := PinotColumnHandle{columnName=studentID, dataType=integer} score := PinotColumnHandle{columnName=score, dataType=real} firstname := PinotColumnHandle{columnName=firstName, dataType=varchar} gender := PinotColumnHandle{columnName=gender, dataType=varchar} subject := PinotColumnHandle{columnName=subject, dataType=varchar} timestampinepoch := PinotColumnHandle{columnName=timestampInEpoch, dataType=bigint} lastname := PinotColumnHandle{columnName=lastName, dataType=varchar}
explain select * from transcript limit 10
x
I think it’s because of one server didn’t respond
c
But I found that, different select got different null server.. SQL Error [65536]: Query failed (#20210720_081531_00058_844up): null value in entry: Server_sj1-pinot-server-23_8098=null
Sometimes: SQL Error [65536]: Query failed (#20210720_081609_00059_844up): null value in entry: Server_sj1-pinot-server-21_8098=null
Sometimes: SQL Error [65536]: Query failed (#20210720_081635_00061_844up): null value in entry: Server_sj1-pinot-server-25_8098=null
Maybe presto just get the hostname but it’s ipaddress?
x
hmm, I think the hostname is ok
it’s resolving correct
but it’s getting empty response from the server
c
Yes I configured all the hostname and ipaddress in pinot cluster /etc/hosts, but presto is manager by other team without this /etc/hosts mapping? maybe presto don’t know what’s the ipaddress of the hostname
x
hmm, can you try to increase number of segments per split
default it’s one segment per split
also there is a config called limit-large-for-broker or similar
c
Should this configured in pinot or in presto?
x
which can be used for make this query goes to pinot-broker directly so it won’t query pinot servers
I think @Dharak Kharod also saw this issue previously
c
Thx @Xiang Fu Seems we can wait for @Dharak Kharod replay
d
Hi Yes, we have been seeing this, I will be looking into it this week, will update as soon as I find something
x
Thanks @Dharak Kharod can you create a github issue for this
c
Hi @Xiang Fu @Dharak Kharod It seems presto pinot0.6.0 libraries is not adapter to pinot 0.8.0 successfully
This query can work : SELECT * FROM sj1_pinot.default.“select * from transcript”;
x
For that you need to add a data table version to v2 in Pinot servers
c
Sorry I don’t know what;s v2 means?
image.png
d
Hi I also looked into it and seems like the data table returned from the server is null, and we also started noticing it with the version upgrade to 0.7.1 , is the data table version upgraded in 0.7.1?
created an issue on the pinto github for now https://github.com/apache/incubator-pinot/issues/7183 , fix should most probably be in the presto-pinot driver
c
Thx @Dharak Kharod 👍
d
@Xiang Fu we reverted the version on the server and query works.. did the datatable version change between 0.6 and 0.7.1 ? @Ting Chen
x
Yes. There is a config change you can put to let new Pinot servers use old data tables
c
Hi @Xiang Fu if I config this: pinot.server.instance.currentDataTableVersion=2 will pinot can work also in our prod env?
x
Yes
c
Thx @Xiang Fu
But if there have some other affects like performance?
or it’s just a version num?
x
No impact from the upgrade since you were on v2 before
V3/v4 gives better perf than v2
c
OK Thx @Xiang Fu