Hi, I am executing a inner join query in Presto bu...
# troubleshootinga
Arpit
11/17/2021, 4:31 PMHi, I am executing a inner join query in Presto but getting below error :
Error when hitting host with Pinot query " select validfrom, Id, InsertTimsttamp from trade_realtime where (id = '1234') limit 2147483647
My original query is like this:
select a.Id, max(a.ValidFrom) as MaxValidFrom, a.InsertTimeStamp from mypinotcluster.default.trade a inner join ( select Id, Max(InsertTimeStamp) as MaxInsertTime from mypinotcluster.default.trade group by Id ) b on a.Id = b.Id and a.InsertTimeStamp = b.MaxInsertTime AND a.Id='4-467125-467125
-0-50' group by a.Id, a.InsertTimeStamp LIMIT 20;
Looks like Presto is computing the result in memory instead of executing in Pinot. Any ideas how can I make it work?
m
Mayank
11/17/2021, 11:17 PM
@Arpit The Presto-Pinot tries to push as much computation down to Pinot as Pinot can support. From what you are describing it seems it is trying to only push down the filter to Pinot and get all records that match that filter (hence INT.MAX_VALUE as the limit).
Mayank
11/17/2021, 11:18 PM
Can you run
select count(*) from trade_realtime where (id = '1234')a
Arpit
11/18/2021, 9:30 AMThere are just 14 records
Arpit
11/18/2021, 2:41 PMI am not able to understand which component is having a problem. My main query with inner join results in this error on Presto
"Error when hitting host <pinot-host> with pinot query "select validfrom, Id, InsertTimsttamp from table where (id='1234') Limit 2147483647"
I can run the above failing query in Pinot directly and it returns 13 rows. Why is Presto giving this error than?
m
Mayank
11/18/2021, 11:55 PM
May be incorrect config
<pinot-host>Mayank
11/18/2021, 11:56 PM
From the connector code, I see it should have logged a valid host:
Copy code
catch (PinotScatterGatherQueryClient.PinotException pe) {
throw new PinotException(PINOT_ERROR_CODE_MAP.getOrDefault(pe.getErrorCode(), PINOT_UNCLASSIFIED_ERROR), Optional.of(pql), String.format("Error when hitting host %s", host), pe);
}a
Arpit
11/21/2021, 10:29 PMThat was just a place holder for the actual hostname. The actual error message has the correct hostname of Pinot controller
m
Mayank
11/22/2021, 3:11 PM
@Xiang Fu any idea
x
Xiang Fu
11/22/2021, 6:25 PM
hi
Xiang Fu
11/22/2021, 6:26 PM
did you get any log from presto coordinator ?
a
Arpit
11/22/2021, 6:26 PMNeed to check once
x
Xiang Fu
11/22/2021, 6:26 PM
for pinot, which version are you running
a
Arpit
11/22/2021, 6:26 PM0.8.0
x
Xiang Fu
11/22/2021, 6:27 PM
got it, are you running the helmchart and has this config for pinot server: https://github.com/apache/pinot/blob/master/kubernetes/helm/pinot/values.yaml#L299
Xiang Fu
11/22/2021, 6:28 PM
presto is still on pinot 0.6.0 lib, so there could be a protocol issue, we need to make pinot backward compatible
Xiang Fu
11/22/2021, 6:28 PM
we are working on getting presto side up to 0.8.0
a
Arpit
11/22/2021, 6:29 PMOhh ok