Apologies if this is the wrong channel but I've ha...
# general
j
Apologies if this is the wrong channel but I've had a question for the last few weeks and can't seem to find any clear answers. Put simply, is there anything that prevents Pinot from being used as a general-purpose CQRS read store? Not as a source of truth, but rather a place where the current state of some item can live for read-only, eventually-consistent access after making its way through, say, Kafka and the various consumers that build the current state for that item. The availability, low latency, and even the ability to do certain joins seem to make it a perfect fit but every time I see anything about Pinot use cases, it's always in the context of analytics. Is this beyond the scope of what Pinot is intended for? If so, I'm interested to understand the limitation. Thanks!
m
Yes, Pinot is primarily an analytics store and not a general purpose eventually consistent database. As in the query engine is highly optimized for analytics use cases.
k
Pinot will work but it will not be as good as a key value store.. is there any reason existing key value stores won’t work.. Another way to think is what’s the read side query pattern.. is it just accessing just one row or multiple rows
m
@Josh Black If you base data has a timestamp and you are wanting to use Pinot for aggregating upserts and getting the results it will work. If as above you are only wanting to do single row queries it may not be as performant as a key value store. If you want transactions involving multiple rows or table it is not suitable. If you want to aggregate the base data by some value or set of values such as group all activity by account or by large grained object then Pinot may be a good store for turning CQRS type streams into a stateful aggregated view of your entities.