Hey team, is tiered storage using cloud storage available in OSS Pinot? Also, in the blog Introducing Tiered Storage in Apache Pinot it mentioned a "decoupled system", which decouples storage and compute, was that achieved by configuring Pinot, or it was some other database?

@Chris Hao The Tiered Storage solution with deepstore as a tier is only available in enterprise version. In terms of decoupling - it is entirely Pinot based system with deep-store as the cold tier. However, it is much more than just lazy loading, the engine employs a lot of smart techniques (eg prefetching, pinning sparse indexes, etc) to drastically improve performance as compared to a lazy loading approach.

In terms of decoupling - it is entirely Pinot based system with deep-store as the cold tier.

got it thanks, is this decoupled architecture also only available in enterprise version?

Yes. What is available in OSS is: tiering between SSD and HDD.

sounds good, thank you

if you’re referring to the

decoupled system

as mentioned here

We then put the same data on a decoupled system. That brought down the local storage utilization from 500G to 0 and added marginal cost for storing the same data on cloud storage (roughly $10/month). But now the same queries have much higher latencies (see column Decoupled)

then that was Presto