https://pinot.apache.org/ logo
Join Slack
Powered by
hi Recently I was working with pinot group by c...
# general
j
hi Recently I was working with pinot group by command . I have loaded 3 million data in data in the table. I executed group by query with limit. If the limit is above 1 lakh, the query returns only 1 lakh records. As per documentations I have changed query limit over riding property to fix this limit issue but it didn't work. The issue is attached as screenshot below. Thanks in advance for the help!
k
Looks like it did not reach the group by limit
What is the cardinality of MSISDN
j
image.png
column.MSISDN.cardinality = 1000000
select MSISDN from instant_offline LIMIT 1000000 This query works fine. The only issue with group by queries
k
Why do you say there is issue with group by? It will return only 1million groups since the cardinality is 1M
Oh sorry, your limit is 100k
What’s the variable you set?
j
pinot-broker.conf pinot.broker.enable.query.limit.override=true pinot.broker.query.response.limit=2147483647
hi any solution regarding this issue
k
This is not the right setting
n
Which documentation did you follow?
the setting is 
pinot.server.query.executor.num.groups.limit
@Jackie please confirm that this setting is sufficient ^^ ?
j
Yes, please set 
pinot.server.query.executor.num.groups.limit
to a higher value
Default is 100K, if you have only 1 segment, then it will only return 100K groups
✅ 1
j
@Kishore G @Neha Pawar @Jackie This configuration is working. Thank you for the support.
Open in Slack
PreviousNext
Powered by