This message was deleted.
# generals
Slackbot
05/10/2023, 7:36 AMThis message was deleted.
f
Felix GV
05/10/2023, 11:34 AMWe have a few JMH benchmarks checked into the repo. Besides that the rest of our performance-related activities aren't in open source. We have some environment where we can generate synthetic traffic (been trying to convince @Min Huang to open source the synthetic traffic generation app). But the ultimate test is the production workloads... we take flamegraphs from clients or clusters that are struggling and then optimize the hot paths.
e
Enrico Olivelli
05/10/2023, 12:59 PMMakes sense
Enrico Olivelli
05/10/2023, 1:00 PMdo you have some client to run benchmarks ? especially for the read workflow
d
Dave Fisher
05/10/2023, 3:02 PMStoring bookmark to https://github.com/openjdk/jmh
m
Min Huang
05/10/2023, 10:47 PM@Felix GV I shared the same vision; just need to find the time to move them, and test in non-LI environment.
🤝 1
Min Huang
05/10/2023, 10:48 PMRead traffic might be easy; as for write traffic, they currently depends on Hadoop and Samza; I am not sure how to generate those traffic without spinning up Hadoop and Samza first.
f
Felix GV
05/10/2023, 11:56 PMYep, let's start with reads then.
Felix GV
05/10/2023, 11:57 PMThen maybe we can throw in @Nisarg Thakkar's online producer in there too, to have synthetic R/W traffic from a single app, no Samza or Hadoop needed.
Felix GV
05/10/2023, 11:57 PMI assume he'll probably want to add it there anyway.
e
Enrico Olivelli
05/11/2023, 9:53 AMNow we also have Pulsar to generate load 🙂
At DataStax we have plenty of tools to generate Traffic on Pulsar
Enrico Olivelli
05/11/2023, 9:54 AMwe will probably start filling in a Venice cluster from Pulsar and then perform random reads to Venice and measure latency
f
Felix GV
05/11/2023, 12:15 PMLooking forward to hearing your feedback on that! There are some important RocksDB knobs that can be tuned for cost vs performance. Happy to dive into it more when you get there.
4 Views