https://pinot.apache.org/ logo
#general
Title
# general
p

Pavel Stejskal

02/11/2022, 6:04 PM
Hello! Can somebody share some performance characteristics for realtime tables? How fast can server ingest from kafka, how to scale via partitions etc? Thanks
m

Mayank

02/11/2022, 6:17 PM
Hello, some stats from production deployments: a 16core 64GB instance has handled 100k events per second (can go higher/lower depending on schema, transform function, etc).
p

Pavel Stejskal

02/11/2022, 6:30 PM
That's great. What is scaling options? More real time servers for each kafka partition? Customer is targeting to peak 1-1.2 mio/s with relatively simple structure, cca 16 columns. Is this capability linearly scalable or whats is pattern here? Thank you
k

Kishore G

02/11/2022, 6:41 PM
what is mio?
p

Pavel Stejskal

02/11/2022, 6:41 PM
Million
m

Mayank

02/11/2022, 7:35 PM
For scaling you can have multiple realtime servers, and each consuming a subset of partitions
p

Pavel Stejskal

02/11/2022, 7:36 PM
👍👍 thank you
👍 1
Exact model like with spark
s

Subbu Subramaniam

02/12/2022, 12:32 AM
@User you can watch this one:

https://www.youtube.com/watch?v=nf11Q97-d8k

👍 1