Hi, I want to restart an Apache Druid cluster cons...
# general
а
Hi, I want to restart an Apache Druid cluster consisting of 200 Historical nodes (approximately 800TB of data). How can I ensure that after the restart, the Historical nodes will continue using the segments stored in their local caches rather than reloading them from Deep Storage? Is there a specific time window within which I must restart each Historical node to avoid triggering segment reloads from Deep Storage? Could pausing the Coordinator help prevent unnecessary segment reloads? Additionally, the Main nodes will need to be restarted as well—the entire cluster will be restarted sequentially: Main nodes first, then Query nodes, and finally Data nodes.
i̇
If your aim zero-downtime, I recommend reviewing this document, which, while focused on rolling updates, is also applicable to restarts: https://druid.apache.org/docs/latest/operations/rolling-updates/ If your concern is only deep storage reload, you can shut down all historicals at once. With no historicals available, balancing won’t occur. Alternatively, you can restart them quickly one by one to minimize balancing. I’m not sure if stopping coordinator will help.
🙌 1
k
You can pause the coordinator so that no rebalancing happens during the restart
🙌 1
during this time any new segments created won't be loaded to historicals