Hi everyone, Will Pinot handles HTAP workloads?
# generala
Ajay Babu
09/27/2023, 2:32 PMHi everyone, Will Pinot handles HTAP workloads?
k
Kishore G
09/27/2023, 3:18 PM
HTAP is a bit overloaded marketing term.. there are many aspects to it. It will be great if you can share the usecase along with the requirements.
a
Ajay Babu
09/27/2023, 3:34 PMThanks for reply
Ajay Babu
09/27/2023, 3:36 PMThe use case is I have a customer base of 5 million and each customer will have ~ 50 transactions / day and I should maintain last 180 days of history, So dataset becomes ~ 45 billion records and size ~ 144 terabytes. Here I am expecting a Pinot should support aggregation query with where as
1.customer_id=? and date >=? and date <? as OLTP / random lookup query I mean, expecting 2k QPS.
2.date >=? and date <? as OLAP / analytical query I mean.
Will Pinot support this both workloads on common set of data.
Ajay Babu
09/27/2023, 3:43 PMExample: Like in Hadoop this both workloads are deliver by two independent components
1. HBase for OLTP
2. Hive for OLAP
Ajay Babu
09/27/2023, 3:46 PMSo here data becomes duplicate, cost extra hardware and more maintenance, So expecting how Pinot will solve similar issue
Ajay Babu
09/28/2023, 4:53 AMAny update?
k
Kishore G
09/28/2023, 5:17 AM
Thanks for the details. Pinot has the right indices to support both these patterns
a
Ajay Babu
09/28/2023, 6:47 AMPinot JDBC driver support this workloads? Actually my java application was running on wildfly server and uses it's connection pool to read data from database. So same approach can follow for when Pinot integration?
k
Kishore G
09/28/2023, 2:17 PM
yes. jdbc driver supports it
a
Ajay Babu
09/28/2023, 3:02 PMI am more interested to know how Pinot solving Point Lookup(OLTP) workloads, generally column-oriented stores are suitable for OLAP workloads.
Ajay Babu
09/28/2023, 3:06 PMCan I get any benchmark report which contains achieved QPS w.r.t data and hardware .
k
Kishore G
09/28/2023, 3:35 PM
I can introduce you to Uber folks and you can talk to them
4 Views