Our data has more than 400 different dimensions. Cube only has 25 of them, but we are planning to increase it, We are aware that adding a new dimension would increase volume with Cardinality of the new dimension (in the worst case). Is there a recommendation on the number of the dimensions too ?? As in how many dimensions I can add around the "group by".

its the cardinality product of the group columns. By default we limit that to 100k in the group by query. (note this is in the actual query, not based on actual data)

thank you, that's helpful. I did see that Pinot supports Dictionary encoding and we have lots of dimensions with low cardinality like os-version, os-type, ips , state, country, segment, cohort , seasonality, etc

yes.

Yes, low cardinality columns will compress very well.

@Kishore G what happens if the cardinality of a column being used for group by is > 100K? If the query has an order by, will it use a priority queue to keep around the (approximate) top results?

yes

So unless the data is weirdly skewed, if our LIMIT is something significantly lower (like 1000) then the final results should be exact, or nearly exact.

yes

OK, thanks. What if it’s something like

select memberId,sum(views) from T group by memberId order by sum(views) desc limit 1000

Here the “top” results are ones with the smallest number of views. Will the priority queue use the order by information to correctly keep the memberId groups with the smallest sum?

priority queue is setup based on the order by clause