https://pinot.apache.org/ logo
j

jose farfan

02/13/2021, 3:38 AM
but : select * from transaction_line limit 2147483645 it is working
k

Ken Krugler

02/13/2021, 3:48 AM
I have to assume somewhere in the code is an
int val = <sql query limit> + 1
, or equivalent, that overflows when you pass Integer.MAX_VALUE as the limit. But why would you want to use such a huge limit? 🙂
j

jose farfan

02/13/2021, 3:57 AM
Hi
The problem is with this query in presto
with lastValue as ( select  player_nr,  *rank*() OVER (PARTITION BY player_nr,id           ORDER BY processTime DESC) AS rnk FROM pinot."default".transaction_line  ) select *count*(*) from lastvalue  where rnk=1 limit 100000;
SQL Error [84213860]: Query failed (#20210213_035642_00049_23fka): Error when hitting host Server_172.19.0.7_8098 with pinot query "SELECT player_nr, processTime, id FROM transaction_line_REALTIME LIMIT 2147483647"
then I getting this message
I am trying to update this parameter pinot.broker.enable.query.limit.override to a small values, but it is not creating the effect that I want
I have used the api to update:
Copy code
{
  "allowParticipantAutoJoin": "true",
  "enable.case.insensitive": "false",
  "pinot.broker.enable.query.limit.override": "10000000",
  "default.hyperloglog.log2m": "8"
}
k

Ken Krugler

02/13/2021, 4:04 AM
I haven’t tried using the API to update broker settings, sorry - but changing in the config has worked for me (then restart the broker processes)
k

Kishore G

02/13/2021, 4:10 AM
Ah if you are using with presto then use the streaming api
@Xiang Fu will be able to point to the instructions to enable that
That will allow full scan in parallel
j

jose farfan

02/13/2021, 4:29 AM
Hi @Ken Krugler, I did the change, and a broker.conf file with this values
Copy code
pinot.broker.enable.query.limit.override = true
pinot.broker.query.response.limit = 10000000
now the query console of pinot is working with this query: SELECT player_nr, processTime, id FROM transaction_line_REALTIME LIMIT 2147483647
but, when I use prestodb, I get the same error
hi, at the end I fixed, I have changed the configuration of presto for pinot, and add this line to pinot file the catalog folder
pinot.forbid-broker-queries:true
And now it is working,
Thx Kishore, Ken for your help
have a nice weekend
k

Kishore G

02/13/2021, 4:52 AM
You too.. btw you don’t want to forbid broker queries for everything
We will help you next week with the right config
j

jose farfan

02/13/2021, 5:02 AM
thx
x

Xiang Fu

02/13/2021, 9:44 PM
to enable streaming connector, in presto, please config :
Copy code
pinot.use-streaming-for-segment-queries=true
in pinot you need to configure:
Copy code
pinot.server.grpc.enable=true
pinot.server.grpc.port=8090
also you need to make the pinot change first then presto changes
j

jose farfan

02/14/2021, 8:23 AM
hi, many thxs, I will try asap