https://pinot.apache.org/ logo
#general
Title
# general
j

Jaromir Hamala

03/02/2022, 8:10 AM
Hello, congratz on the tiered storage! I'm reading the announcement and it says: Note that this is not implemented as lazy-loading - Pinot servers directly query data on the cloud and are never downloading the entire segments locally. May I ask how does it work? I know close to nothing about S3, but I believe it's a dummy blob-store. You have to download blobs with segments before querying them, don't you? Am I missing anything? Thanks for any hint!
m

Mark Needham

03/02/2022, 11:04 AM
cc @User @User
👍 1
k

Kishore G

03/02/2022, 2:33 PM
We will publish another blog on the details but at a high level S3 has api where you can read a part of the blob and we leverage that
l

Luis Fernandez

03/02/2022, 2:43 PM
Where do we get this announcements?
k

Kishore G

03/02/2022, 2:50 PM
l

Luis Fernandez

03/02/2022, 2:53 PM
😮
this will definitely be beneficial for our use case thank you!
🍷 5
j

Jaromir Hamala

03/02/2022, 3:16 PM
If I understand it right Apache Pinot won't copy the whole blob, but just some parts of it. Thank you!
k

Kishore G

03/02/2022, 3:52 PM
thats right, only the parts that are needed to process the query.
👍 1