Apache Pinot

Question, I just watched this awesome <https://www.youtube.com/watch?v=YUJXwzN2dok|video> about Tiered Storage in StarTree (thanks Neha!). I see the open-source Tiered Storage <https://docs.pinot.apache.org/v/release-0.10.0/operators/operating-pinot/tiered-storage|docs> mention using a DFS such as S3:
&gt; it might become desirable to keep only the recent time ranges on SSDs, and keep the less frequently queried ones on cheaper nodes such as HDDs or a DFS such as S3.
Yet lower in the docs, it appears `storageType` only seems to support `pinot_server` (which seems consistent with current <https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/tier/TierFactory.java#L58-L64|code>).

Is Tiered Storage with cloud-based blob store supported with OS pinot or is it a StarTree proprietary feature?

In OSS, the support for tiering is on SSD vs HDD. Rather, tiers are attached to different servers 

Thanks for the quick response Mayank. Makes sense and is consistent with my take away from reading the docs page / source code.

<@U03HC6EGURW> I’d even say that OSS support isn’t really tiered storage in the way that people think of that term, but rather multi volume support.

<https://startree.ai/blog/multi-volume-support-in-apache-pinot>

It can be done differently in OS Pinot (see `tierConfigs`) in the article above:
1. Multi-volume support as the article above describes. `"name": "hotTier",` therefore points to respective volume.
2. Multi-tenant support, where `"serverTag": "DefaultTenant_OFFLINE"` may point to a tenant that has HDD-powered volume attached. 

Personally I would probably suggest evaluating the second approach, as having different tenant allows the "colder" data to be naturally MMAPed to available page cache, thus removing "competition" for physical memory between hot and cold tiers.

Please also see <https://github.com/apache/pinot/pull/10746> for direct tier assignment on ingestion.

Speaking of proprietary feature of StarTree (using DeepStore "directly"), are there any articles that describe implementation and evaluate performance? As naively reading segments from S3 upon request may be slow and costly.

did you already read Neha’s blog post? <https://startree.ai/blog/introducing-tiered-storage>

AFAIK it’s doing some quite clever reading of the data in S3 to get a decent balance

&gt; Pinot servers directly query data on the cloud and are never downloading the entire segments locally
Thanks. This is interesting as I see that v3 segment format is essentially a monolithic file.