Hey druid folks! My team is facing an issue where ...
# general
u
Hey druid folks! My team is facing an issue where due to some scaling requirements, we are having to scale up our MMs for our morning ingestions. During this time we also poll the coordinator segment availability API consistently as we have other workflows. Currently running 1Coordinator, 1 Overlord 2-4MMs 2Historicals 1Router 1 Broker. During this time we are seeing API Delays due to increased load, even a coordinator restart. Would it be beneficial to scale out (Horizontal scale) our coordinator service? We have only till now vertically scaled this service. Would there be any configuration between multiple coords that needs to be managed. Hosted on k8s
j
Hi Utkarsh, I am not an expert in this area of Druid, but there is only one "leader" for Coordinator and Overlord in the cluster at any given time, regardless of how many Master nodes you have ... so not sure a horizontal scaling would do anything here. How big are you servers that Coordinator and Overlord run on, and how big are those JVMs?
u
Hi again John! We were able to resolve this issue by figuring out that the underlying RDS of our system was the choke point. By scaling that up this issue was resolved.