Does anyone have suggestions for optimizing an imp...
# general
p
Does anyone have suggestions for optimizing an impressions table that needs to return a distinct count of user_ids over multiple months? We are filtering and grouping on
page_name
šŸ‘€ 1
j
Does approximating the results work if so this should help (https://docs.pinot.apache.org/configuration-reference/functions/distinctcountthetasketch)
Assuming page_name already has an inverted index for filtering
p
Jayesh did not have an inverted index. That solved the issue! Thank you
šŸ™Œ 1
k
If you partition on userid, you can use partitioned distinct count
m
Yea, partitioned version will be accurate and fast. If you are ok with approximations, I’d recommend HLL over theta sketches purely for count distinct. You can use HLL with StarTree index to make it even faster
šŸ‘€ 1