Hi all - I am really interested in Pinot and tooling like it which is enabling simpler development for building customer-facing analytical applications, especially cases where you are moving beyond a dashboard and want to really embed data into the application workflow. I have spent a few months surveying a lot of the different tools and approaches that are coming together to make this easier, including Pinot. I wrote up some of what I have seen here, in case it is of interest to anyone: https://towardsdatascience.com/the-analytical-application-stack-eead8ce6b70

Thanks @User for sharing, it is a very well written article.

Great article @User. I always keep thinking about how a generic stack would look like.

The reason I separated caching is because I see a lot of instances where companies are forced to introduce pure caching layers, primarily to offset deficiencies in the underlying storage engine. I tried to highlight in the article that in many cases, the DB layer itself may be sufficient to either obviate the need for complex caching (e.g. it is fast enough that caching is irrelevant) or natively include strong caching, in which case a secondary caching layer is indeed likely irrelavent

@User nicely written article, feels like you have been in my head 🙂 , we are starting to see similar use-cases proliferating and are working on a few. I will read more deeply plus add our experience and give you a more detailed feedback

I think the explanation makes total sense. I was bit misled by the diagram.. it makes it look like caching layer calls the underlying data layer