Hello! I need some opinion with my configuration. I have more that 16 datasources (Kafka streaming) and I want to run at least 1 task per datasource to make my data almost "live". My machine has 16 CPUs and according to documentation capacity of middle manager should equals number of CPUs on the machine - 1. What is the best solution to increase my capacity, to be able to run more that 16 tasks? I tried to set manually a larger number of capacity, but then my tasks started to fail.

I thought (Num of CPU - 1) was only a recommendation. Does it actually block from configuring a higher number of MM workers (

druid.worker.capacity

) than (Num of CPU - 1)? If the data velocity is not high, then the CPU would be idle for the most part so it doesn't make sense to block additional processes from starting.

When I set capacity higher than Num of CPU, my historical terminates and all tasks are failing

Historical dying might indicate a memory shortage - have you considered the memory consumed by each task + historical memory vs the memory available in the system?

Ok, I think I should discover that, thanks