https://pinot.apache.org/ logo
#general
Title
# general
h

Hamza

06/14/2021, 3:01 PM
Hello, I'm runnning Pinot in K8s and I have a job that creates my table, my schema and another job that does the ingestion from a GCS storage. This last job creates segments and store them in a GCS bucket. Is there a way for later runs to load these segments directly from the folder without recreating them ?
m

Mayank

06/14/2021, 4:11 PM
You can choose the job type that only does the push https://docs.pinot.apache.org/configuration-reference/job-specification
h

Hamza

06/14/2021, 4:12 PM
that works well, thanks !
m

Mayank

06/14/2021, 4:13 PM
You probably want to use metadata+uri push since you are using deepstore. This way only metadata will be pushed to controller, and servers will download it from deepstore
h

Hamza

06/16/2021, 12:33 PM
So should I use SegmentUriPush or SegmentMetadataPush as Jobtype ?
m

Mayank

06/16/2021, 1:20 PM
Metadata push
h

Hamza

06/16/2021, 1:51 PM
Ok Thanks