Hey all , wanted a general opinion . I recently started working with druid . our compaction service config is like maxRows = 5 million , but we also hash by OrgId . it was working well but we now have a big org , adding around 5 billion logs a week . and their spikes are also huge like 20 million in one hour and maybe 200 million in other . so basically wanted to optimize for skewness in this case . I was thinking of two things , first lowering segment partition from day to hour and second adding a secondary dimension like Control Plane Id would love to know if anyone had similiar experiences and has better explored any better options . also raised an Issue for a feat I would want to add for warning on such skewed cases : https://github.com/apache/druid/issues/19573 , would love views on that as well .

PR sounds good to me but is out of my ballpark. But, adding another column (ideally high-cardinality) sounds like a good idea and should help.