Hello Druid users,
Just wanted to tip you of this nifty little feature in Kafka ingestion. You can configure Kafka Supervisor to stay idle when data on stream is sparse and is inactive for a while and no index tasks are spun during this time. When data arrives on stream again, Supervisor will create index tasks as usual. This saves computing cost without effecting performance when paired up with autoScalerConfig. You can find more information on the configuration here https://druid.apache.org/docs/latest/development/extensions-core/kafka-supervisor-reference.html#kafkasupervisorioconfig