How many are you able to ingest now?
Whats the size of the records?
Are you splitting the ingestion into multiple tasks?
Do your middle managers use SSD disks?
c
Chandu
06/21/2023, 11:54 PM
We able to achieve the ingestion rate 130k/sec.
Number records 200 billion.
We have splitted the ingestion into multiple tasks.
I don't think using ssd disks for middle manager but the druid cluster deployed in k8s.
c
churro
06/22/2023, 2:22 AM
use mm-less and increase partitions. spread the load