This message was deleted.

It is common to deploy Historicals as a stateful set in K8s. This is precisely to avoid having to download segments into the cache in the case of temporary outages/restarts/upgrades of Historical pods.

OK, thanks. We are using the druid operator (https://github.com/datainfrahq/druid-operator) and configuring out from its examples which did not have any volume mounts in the historical servers (at least in the examples we saw when we set it up) so it was unclear about the segment cache. If we can set up per pod persistent volumes, then that will definitely speed things up on restart as well as take pressure off ephemeral storage on the K8s nodes 🙂

For deepstorage you can use object storage, use pvc's ( ebs / ssd's ) for historical's.

Right, which we will when it goes to prod but for now we have been doing a POC with Druid so haven't set up something like S3 for deep storage.

yeah even minio works well, we run e2e tests on operator with minio

Hi I have deployed Apache Druid on kubernetes. I need to connect to S3 and I have enabled S3 connection via druid.extensions.loadList=["druid-s3-extensions"]] Do I need to add anything else? When I make the connection with URI, plus acceskey and secretkey I get this error: What else do I need to add? Thank you for your help Hi I have deployed Apache Druid on kubernetes. I need to connect to S3 and I have enabled S3 connection via druid.extensions.loadList=["druid-s3-extensions"]] Do I need to add anything else? When I make the connection with URI, plus acceskey and secretkey I get this error: What else do I need to add? Thank you for your help