This message was deleted.

this is not answering your question directly, but have you looked at druid's new json column feature? https://druid.apache.org/docs/latest/querying/nested-columns.html

I’ve read about json column some time ago, but I wasnt considering it as I’m hoping to run queries like

select qualifiers, count(*), count(distinct id) uniqueIds
from table
where qualifiers = 'type1=value1'

[type1=value1, type2=value2, type17=value17]  | 17 | 10
[type1=value1, type3=value3, type17=value34]  |  3 |  3
[type1=value1, type8=value2, type17=value34]  | 11 | 10

with json columns that'd be

SELECT qualifiers, count(*), count(distinct id) uniqueIds
FROM table
WHERE JSON_VALUE(qualifiers, '$.type1') = 'value1'`

interesting, I was expecting MV approach to be like much faster due to the bitmap indexing.

hope it works well for you! yeah, we put a lot of effort into making the JSON column implementation in many cases just as speedy as not using JSON columns. since the API is so nice we want people to be able to use it without worrying about perf

Does it work with roll ups and LATEST functions?

yep: https://druid.apache.org/docs/latest/multi-stage-query/concepts.html#rollup

It’s working very well so far! Haven’t tried under heavy load or fancy queries yet, but so far so good

nice 👍