Hello, congratz on the tiered storage!
I'm reading the announcement and it says: Note that this is not implemented as lazy-loading - Pinot servers directly query data on the cloud and are never downloading the entire segments locally.
May I ask how does it work? I know close to nothing about S3, but I believe it's a dummy blob-store. You have to download blobs with segments before querying them, don't you? Am I missing anything? Thanks for any hint!
03/02/2022, 11:04 AM
03/02/2022, 2:33 PM
We will publish another blog on the details but at a high level S3 has api where you can read a part of the blob and we leverage that