i wanna do some benchmarking with this combination...
# generale
Erjan G.
04/26/2021, 5:14 PMi wanna do some benchmarking with this combination: cassandra as storage + analytics on pinot
m
Mayank
04/26/2021, 5:17 PM
One of the powers of Pinot is its indexing techniques (its own storage component).
Mayank
04/26/2021, 5:17 PM
If you can provide more details around your use case, perhaps we can suggest better alternatives?
e
Erjan G.
04/26/2021, 5:51 PMhmm in fact i m only learning cassandra and looking for ways to integrate it with any OLAP system.
i dont have any use case now, just wondering about easy of integration btw cassandra and pinot. by the way we are based in Kazakstan lol 🙂
m
Mayank
04/26/2021, 5:53 PM
Nice that Pinot is also reaching Kazakstan 🙂. So typically, for Pinot, you ingest data either from Realtime streams (eg Kafka) and/or batch (ie offline data push).
Mayank
04/26/2021, 5:54 PM
As I mentioned, Pinot is fast partly because of its own storage + execution engines. Using another storage system is probably not going to get what you want. I am not sure if I see the value in using Cassandra as storage and Pinot as query execution.
Mayank
04/26/2021, 5:57 PM
If your end goal is analytics, then better to use just Pinot by itself.
e
Erjan G.
04/26/2021, 5:58 PM@User u mean integration of casandra and pinot is not a good idea? but where does pinot get data from? or pinot just connects to whatever place where the data is stored?(s3, hdfs)
m
Mayank
04/26/2021, 5:59 PM
I am saying I am unsure what the value add is for connecting the two (unless you have strong business case).
e
Erjan G.
04/26/2021, 5:59 PMto me Casandra is storage, so i assumed that any other apache tool (druid, pinot) can simply connect to it and i can run my olap queries directly from casandra... wrong?
m
Mayank
04/26/2021, 5:59 PM
Pinot is also storage.
Mayank
04/26/2021, 5:59 PM
The question is do you need OLTP storage, or are you good with OLAP storage
e
Erjan G.
04/26/2021, 5:59 PM@User oh ok, i was just trying to integrate them for the sake of seeing how well they glue together.
m
Mayank
04/26/2021, 6:00 PM
If you need OLTP, then many folks use flows like OLTP -> debezium (or similar systems) to push changes to messaging system (kafka) -> Pinot
e
Erjan G.
04/26/2021, 6:00 PMyeah i should probalby focus on actual biz case thx mayank
m
Mayank
04/26/2021, 6:00 PM
yeah, that would define your requirements, and help us suggest better solutions / usage of Pinot
🙌 1
k
Kishore G
04/26/2021, 6:15 PM
Kishore G
04/26/2021, 6:16 PM
you can replace MySQL in that diagram with any other storage - Cassandra, MongoDB etc
e
Erjan G.
04/26/2021, 6:27 PM@User hmm thx , where can the kafka connect be placed? in cloud?