Hi, we have Pinot on Kubernetes. We used PVC for s...
# general
n
Hi, we have Pinot on Kubernetes. We used PVC for storing data, but a few days ago, unfortunately, the PVC was mistakenly deleted when all nodes were deleted. Can we use another service for storing data, such as AWS RDS or S3?
t
First of all, that sucks. I’m really sorry for that data loss. I want to hope that things are more or less recoverable.
Second, to my knowledge you can’t use RDS or S3 to store active segments. Even in StarTree’s hosted Pinot—not typically our topic of discussion here!—the tiered storage feature typically puts only some data in S3, with “local storage” (PVCs in your case) holding newer data.
m
@Nikesh Landge I think you are looking for this: https://docs.pinot.apache.org/basics/components/table/segment/deep-store
You need to configure deep-store to ensure data durability.
v
@Nikesh Landge do you use official helm charts?
let me take a look how you can enable deep-store in that chart
r
in the chart this is typically done by setting
controller.data.dir
in the values to
s3://{your-bucket}
🙌 1