Hey @Mohammad Ghazanfar I was researching this a bit and it's going to be a tall order. Have you thought about getting Imply to manage your clusters?
m
Mohammad Ghazanfar
02/06/2024, 6:14 AM
Thanks @Mike Sherman!
Yes, that would be one option I guess.
m
Mike Sherman
02/06/2024, 3:44 PM
The other thing to remember @Mohammad Ghazanfar is that you could get the 30 Druid cluster configuration up and running. Heck, you might even get promoted after doing that! But what then? Who's going to maintain it long term? You would probably be the only one who knew how the clusters worked and you would have to turn it over to someone else at some point...
m
Mohammad Ghazanfar
02/06/2024, 4:13 PM
😂
Mohammad Ghazanfar
02/06/2024, 4:16 PM
We have a ~10 member team for Druid in Confluent.
We already run about 10 Druid clusters. Two of them are too large - have about 400-450 EC2 nodes. We want to break them down to about a dozen smaller and more manageable Druids.