https://pinot.apache.org/ logo
#general
Title
# general
j

John Knapp

04/20/2021, 3:46 AM
Hi, I'm new to Pinot and have a possibly dumb question - with so many features and horizontal scalability, when should one not use Pinot as a general relational database solution?
m

Mayank

04/20/2021, 3:48 AM
Pinot is optimized for performance, but is not a general relational database. For example, it doesn't provide transactions, or support for complex joins/nested queries.
j

John Knapp

04/20/2021, 3:54 AM
Thanks. Do you have any links that talk about the limitations? I know that joins are not supported but presto is supposed to enable that.
m

Mayank

04/20/2021, 3:55 AM
Yes, we close those gaps with Presto
We don't have an exhaustive list of limitations but here are the major ones:
Copy code
1. Non transactional, eventually consistent
2. Does not support full SQL (used with Presto to fill that gap).
3. Not a source of truth
j

John Knapp

04/20/2021, 4:01 AM
I see. Well, it sounds like it should pair well (to use wine terminology) with any other transactional database. Use the transactional DB for read-modify-write and use Pinot for reads so long as eventually consistent is acceptable.
Can it be paired with a no-sql DB like Mongo DB?
d

Daniel Lavoie

04/20/2021, 4:05 AM
If you can leverage a CDC infrastructure to get your changeset to pinot, it does work nicely together
m

Mayank

04/20/2021, 4:07 AM
So the pattern there is to emit the changeset to a messaging system (eg Kafka) which Pinot can consume from
d

Daniel Lavoie

04/20/2021, 4:09 AM
And since Pinot scales well for reads and write, it can be fed by different transactional sources from your system.
j

John Knapp

04/20/2021, 4:12 AM
CDC = change data analysis?
d

Daniel Lavoie

04/20/2021, 4:12 AM
Change Data Capture
j

John Knapp

04/20/2021, 4:14 AM
Any recommended CDC architectures? debezium any good?
m

Mayank

04/20/2021, 4:18 AM
Yeah I have heard folks using that
Debezium is definitely the way to go !
Kenny has another blog coming up with Debezium and Pinot upserts which is even better
j

John Knapp

04/21/2021, 4:20 AM
thanks!