:wave: Wanted to check in about best-practices/con...
# general
b
๐Ÿ‘‹ Wanted to check in about best-practices/configurations when using a deep store with Pinot. I'm interested to see what the best practice is around server storage setup (EBS vs. EC2 local volume) when an S3 deep store is configured for a cluster.
I've found this ๐Ÿงต from about a year ago (Slack Context) that gives the recommendation that if a S3 deep-store is configured, then EBS persistent storage should still be mounted on the pinot servers and to strictly treat the deep-store as a "backup" of all data
I've wanted to see if anyone has gone down the route of using EC2 local storage on servers once a deep-store is used and whether that's possible or recommended.
m
Server storage can either be local or EBS. But would not recommend something like NFS or deep store
b
The current thinking is: S3 for deep-store, possibly EC2 local disks for servers but given what I read in this thread, it makes me think that only EBS is supported for server storage even with a deep-store enabled
m
Deepstore is meant to keep the golden copy of data for disaster recovery. Server disk (ebs or local) is to be thought as servers copy for faster serving. They are not mutually exclusive
b
That makes sense! We'll try out both configurations to see what works best for us! thankyou for the clarification @Mayank !
m
Sure thing