Lari Hotari
09/28/2026, 7:34 AM./gradlew :tests:performance:launcher:run --args='--scenario tests/performance/scenarios/iot-telemetry.yaml'
Linux x86_64 is recommended for measurements. It also works on MacOS (tested with OrbStack). Feedback and improvements are very welcome!
Reporting examples:
β’ Test report
β’ jonoff summary report for application's blocking callsites
β’ Off-cpu flamegraph example for application's blocking callsites
β’ *Grafana*: Integrated and pre-configured to collect metrics during the test runs
β’ VictoriaMetrics, Metrics backend for Grafana, exploring metrics example
β’ VictoriaMetrics, *PromQL* exampleLari Hotari
09/28/2026, 8:16 AMLari Hotari
09/28/2026, 3:29 PMperformance test iot-telemetry-max-rate.yaml with dispatcherMaxReadBatchSize set to 500 and 1000. compare it to the baseline (value 100). Don't do profiling.
Response:
The comparison is running in the background, so there are no results yet. It is 9 runs with no profiling, and the first baseline run started at 18:26. I'll check progress at about 18:52, and I'm
notified as each run starts and finishes.
How it's set up:
- Rounds: 3 rounds of iot-telemetry-max-rate.yaml with dispatcherMaxReadBatchSize set to 100, 500 and 1000. The order rotates each round (100β500β1000, then 1000β100β500, then 500β1000β100) so
that slow drift in the host doesn't favour one value.
- Setting: each run sets the value with --set cluster.brokers.env.dispatcherMaxReadBatchSize=<n>. The baseline sets 100 explicitly, which is also the default in conf/broker.conf.
- Build: all runs use one image (-Pdocker.tag=batchsize), since only a setting changes and the code doesn't.
- Cool-down: each run waits for the CPU to cool to 50 Β°C before it starts.
- Guards: the script stops if thermald becomes active or the disk has less than 15 GB free.
- Where things are: runs are grouped under the names dispatcher-read-batch-{100,500,1000} in the shared reports root. The script is run-batchsize.sh and its log is runs.log, both in the
scratchpad.
Host: the host check first failed 7 of 11 checks: thermald was running, turbo was on, the governor was powersave, and swappiness and huge-page settings were off. I applied the performance-testing
profile with the passwordless configure-perf-test-environment.sh start rule, and all 11 checks then passed. I'll run stop when the runs are done to restore your normal settings. It also wrote
inttest.asyncprofiler.skipPerfEventTuning=true to ~/.gradle/gradle.properties, which stop removes again.
When the runs finish, I'll check each report for correctness: every message delivered, no ordering violations, no throttling. Then I'll give you a table with the median and minβmax of throughput,
publish latency and end-to-end p50/p99 for 500 and 1000 against 100.Lari Hotari
09/28/2026, 3:30 PMLari Hotari
09/28/2026, 7:53 PMAll 9 runs completed without profiling, 3 per setting in rotating order on master f925a25884e6. Every run was valid: all 5,000,000 messages
delivered, with no duplicates, no ordering violations and no thermal throttling. I checked in each run's resolved config that the setting took
effect.
βββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ
β Median (minβmax) β 100 (baseline) β 500 β 1000 β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ€
β Delivered throughput, msg/s β 90,547 (89,280β91,956) β 92,928 (89,738β93,585), +2.6 % β 92,567 (92,432β94,663), +2.2 % β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ€
β Publish p50 β 1,062 ms β 1,037 ms (β2.4 %) β 1,033 ms (β2.7 %) β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ€
β End-to-end p50 β 1,355 ms (1,319β1,390) β 1,348 ms (β0.5 %) β 1,336 ms (β1.4 %) β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ€
β End-to-end p99 β 1,921 ms (1,901β1,951) β 1,921 ms (0 %) β 1,878 ms (β2.2 %) β
βββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ€
β End-to-end p99.9 β 2,023 ms β 2,069 ms (+2.2 %) β 2,022 ms (0 %) β
βββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ
- Throughput: all three runs at 1000 were faster than every baseline run. At 500, one run fell inside the baseline's range. However, the baseline
runs got slower over the session (92.0k, then 90.5k, then 89.3k), so part of the gap may come from timing rather than the setting.
- Latency: the differences are within the spread between runs. At maximum rate, latency mostly reflects the 100,000 messages each producer keeps
in flight, so it follows throughput rather than measuring the dispatcher.
- Next step: if you want a firmer answer, we could do more repetitions and a profiled broker comparison. That would show whether larger reads
actually lower the dispatcher's read count or CPU per message on this workload.Lari Hotari
09/28/2026, 7:54 PM