Hello, I’m looking for support with capacity plan...
# general
s
Hello, I’m looking for support with capacity planning in an on-premise environment. Could you please provide specific formulas or guidelines for these calculations? My setup requires Druid to consume data from Kafka, which has 10 partitions and 2 replicas, at a rate of approximately 1 million events per second (500 MB/sec). The retention policy is 1 month locally (1.3 PB) and 1 year on HDFS. Additionally, there are about 30 to 50 parallel ad hoc queries. I am concerned that if I need to store 2 replicas of the data in Druid, it could significantly increase the storage requirements. I would appreciate a clear plan or approach to handle this.
@Abhishek Balaji Radhakrishnan @John Kowtko - Please could you guide me on this