Hello Druid folks! I had some questions about Druid Historical and Middle manager's (Deployed on K8s using helm) requirement of persistent storage. From my understanding, the reason the storage is there is to serve queries via segments loaded. When a brand new pod is spun up, I see a a loading of all segments via druid logs. Is this the loading of segment map locations from the attached PV to the memory of the historical? Or is it a reload of segments from the S3 True source of segments data? In case its the later, what is the reason for using a persistent storage vs a ephemeral storage? One reason I can think of is that a pod restart can happen which would not require all the data to be reloaded. But when a pod is rollout restarted, and old pods terminate and new ones are loaded and attach to the same PV, why does it still take so long for the pod to start serving queries?

Hi Utkarsh, The "segment loading" you see should be loading segments from deep storage to the Historical's local disk. From there the segment files are direct memory mapped for the processes to access the data for serving queries. There has been some talk of allowing storage to remain intact upon K8s pod restarts, but I don't think that has been implemented as yet (I may be wrong though so if someone else has newer info please let us know).

Hello again John! On that note, does it make sense for the storage used by druid to be of a persistent nature? 1. Is that only to ensure that during a restart the segments remain where expected? Where as during terminate and re add - it will reload segments completely as per your answer 2. Is there any way to use ephemeral storage for historical?