ok, thanks. Let me try and see what difference it makes
Amit Chopra
01/13/2021, 9:49 PM
I tried with no dictionary for medium and high cardinality columns. Though the performance did not improve. It actually became worse 😞.
FYI - @Kishore G@Jackie
j
Jackie
01/13/2021, 10:18 PM
Then the bottleneck for this query is storing and processing all the groups instead of scanning the values. It is more efficient to store dictionary ids comparing to store actual values
a
Amit Chopra
01/13/2021, 10:30 PM
got it, thanks
k
Kishore G
01/13/2021, 10:33 PM
will be good to put these numbers in an issue, there might be some bottlenecks we are not aware of