Hi, I am executing a inner join query in Presto bu...
# troubleshooting
a
Hi, I am executing a inner join query in Presto but getting below error : Error when hitting host with Pinot query " select validfrom, Id, InsertTimsttamp from trade_realtime where (id = '1234') limit 2147483647 My original query is like this: select a.Id, max(a.ValidFrom) as MaxValidFrom, a.InsertTimeStamp from mypinotcluster.default.trade a inner join ( select Id, Max(InsertTimeStamp) as MaxInsertTime from mypinotcluster.default.trade group by Id ) b on a.Id = b.Id and a.InsertTimeStamp = b.MaxInsertTime AND a.Id='4-467125-467125 -0-50' group by a.Id, a.InsertTimeStamp LIMIT 20; Looks like Presto is computing the result in memory instead of executing in Pinot. Any ideas how can I make it work?
m
@Arpit The Presto-Pinot tries to push as much computation down to Pinot as Pinot can support. From what you are describing it seems it is trying to only push down the filter to Pinot and get all records that match that filter (hence INT.MAX_VALUE as the limit).
Can you run
select count(*) from trade_realtime where (id = '1234')
and share the results?
a
There are just 14 records
I am not able to understand which component is having a problem. My main query with inner join results in this error on Presto "Error when hitting host <pinot-host> with pinot query "select validfrom, Id, InsertTimsttamp from table where (id='1234') Limit 2147483647" I can run the above failing query in Pinot directly and it returns 13 rows. Why is Presto giving this error than?
m
May be incorrect config
<pinot-host>
?
From the connector code, I see it should have logged a valid host:
Copy code
catch (PinotScatterGatherQueryClient.PinotException pe) {
            throw new PinotException(PINOT_ERROR_CODE_MAP.getOrDefault(pe.getErrorCode(), PINOT_UNCLASSIFIED_ERROR), Optional.of(pql), String.format("Error when hitting host %s", host), pe);
        }
a
That was just a place holder for the actual hostname. The actual error message has the correct hostname of Pinot controller
m
@Xiang Fu any idea
x
hi
did you get any log from presto coordinator ?
a
Need to check once
x
for pinot, which version are you running
a
0.8.0
x
got it, are you running the helmchart and has this config for pinot server: https://github.com/apache/pinot/blob/master/kubernetes/helm/pinot/values.yaml#L299
presto is still on pinot 0.6.0 lib, so there could be a protocol issue, we need to make pinot backward compatible
we are working on getting presto side up to 0.8.0
a
Ohh ok