Eugene Ramirez
06/16/2021, 3:53 AMback up/restore
feature. I appreciate if anyone can help.
Here is my setup.
Kubernetes: EKS 1.20.4
Pinot version: 0.7.1
So I enable S3 as deep storage based from this link.
Then ingested Parquet data from S3 based on this instruction.
Data loaded fine and I can query the expected data from Pinot.
Next I simulated replacing the cluster, by uninstalling all pods and its related volumes(therefore losing all state) but kept the segment files in s3 segment location(therefore backup is intact in deep store).
Next I reinstalled cluster, and reconfigured the tables. I was expecting that the servers would automatically fetch the segments from deep store as mentioned in previous post, but it does not seem to be happening.
Am i missing a step? Thanks in advance.
https://apache-pinot.slack.com/archives/C011C9JHN7R/p1623336369017900?thread_ts=1623327667.015000&cid=C011C9JHN7RKishore G
Eugene Ramirez
06/16/2021, 4:23 AMKishore G
Eugene Ramirez
06/16/2021, 4:31 AMUploadSegment
command, the parameter segmentDir
requires a local path. This means i have to download the segments first to upload. Is there a way to use the previous cluster’s s3 segment path as source location to new cluster s3 segment path upload?Kishore G
Eugene Ramirez
06/16/2021, 4:39 AMMayank
Eugene Ramirez
06/16/2021, 6:18 AMMayank
Sadim Nadeem
07/01/2021, 9:16 AM