https://pinot.apache.org/ logo
e

Elon

09/09/2020, 6:30 PM
Is all of this going to be in pinot 0.5.0?
t

Ting Chen

09/09/2020, 6:33 PM
e

Elon

09/09/2020, 6:36 PM
Oh great, so if we take latest master and apply this one then we should have everything?
Or do you recommend we just take 0.5.0 and patch this pr?
t

Ting Chen

09/09/2020, 6:37 PM
the latest master has all PRs. Depending on what you need, 0.5.0 has been more thoroughly tested and certified.
👍 1
The integration tests in the PR above also has the necessary config:
(1) a per table config to turn on deepstore by-pass:
segmentsValidationAndRetentionConfig.setPeerSegmentDownloadScheme("http"); // or https
(2) a server config to let the server download segments from deep store.
pinot.server.instance.segment.store.uri
and
pinot.server.storage.factory.class.xxxx
to config the deep store uri and FS class
e

Elon

09/09/2020, 6:45 PM
Sounds great, thanks so much @Ting Chen! I think to be safe we will take 0.5.0 and apply #5857 and follow your advice for the table configs. We use gcs pinot fs. Does that sound like it would work?
t

Ting Chen

09/09/2020, 6:50 PM
at this point, this feature should be still considered experimental -- I have done my best to test in an integration test with all Pinot components.
e

Elon

09/09/2020, 6:50 PM
Sure! Will let you know how it goes. Appreciate all the help!
t

Ting Chen

09/09/2020, 6:50 PM
you may also need to test integrating with GCS.
yes please let me know and I am be happy to help along the way.
👍 1