Hi everyone, I am doing PoC over druid(using S3 batch load for ingestion and then querying through web console) where during the querying - the GroupBy seems to be very slow for dimension with high-cardinality. The whole dataset size is around 25GB with 58M rows where the particular dimension(being grouped by) is expected to occur 1-5 times. Also the queries are run on EC2 with 16vCPU and 64GB memory(3 hosts are deployed as query servers). Does anyone has any inputs to improve the query performance here? TIA

Wait for druid 31 with DART 🚀

What is the improvement expected after druid 31 release(I am currently using version 30.0.0)?

Thanks Karan, will check this.

In addition to DART ... 58M / 5 = 12M ... so the group by is returning ~12M rows? I couple of thoughts on things you could try on the current system: • range partition on the grouping key, may result in less merging work at the Broker level. • If segment scan times are high and you have a lot of historical cpus, try creating more, smaller segments to obtain more parallelism.