Hi community! I’ve created <a GH issue> to start t...
# general
g
Hi community! I’ve created a GH issue to start to discuss about the idea of increasing the Pinot minimum Java version to Java 21. Feel free to participate, specially if you think it may affect you
r
Looking forward to seeing ZGC becoming the default
I'm also looking forward to Java 25 as project Loom and Amber have provided many useful features
g
Zgc can be used right now. We could set it as the default in java 21. That doesn't require upgrading the source level. Loom could be useful for MSE, although we could also use reactive or our coroutines as well. I haven't taken a look at Amber in a while. But I'm afraid supporting Java 25 right now looks like sci fiction 😆
r
I was referring to Generational ZGC
g
My point is that the GC is a runtime-only feature. It can be used right now and we could turn it on by default only when runtime is 21 or higher. The reason we are not doing that is because even though we think gzgc would be ideal for Pinot, we didn't get the chance to properly benchmark
r
In my real-time ingestion performance testing, I'm seeing that Generational ZGC in Java 21 decreased performance by 33%. My heap was only 8GB. But according to my sources, Generational ZGC outperforms G1GC for heaps 32GB and up.
g
have you tried that on a linux os? ZGC is mostly focused on linux and some optimizations (including NUMA) aren’t implemented in Win/Mac
r
Yes, I tested in a k8s cluster running on a VM running Ubuntu. I also tested with larger heaps of 16GB and 32GB, and achieved significantly better throughput with G1 (and some additional JVM options).
g
interesting. In ST we didn’t dedicate actual time to run our own benchmarks. Anyway, remember that is expected to have worse throughput with ZGC, but the improvement should be in latency.
r
I also found that average and max pause times with G1 were sub-second, but multi-second with ZGC.
g
wow
that doesn’t make sense
there has to be something bad in the config
r
Any ideas on what could be bad in the config? I'm using Pinot 1.4, JDK 21, and these JVM options:
jvmOpts: "-Xms32G -Xmx32G -XX:+UseZGC -XX:+ZGenerational -XX:MaxGCPauseMillis=200 -Xlog:gc*:file=/opt/pinot/gc-pinot-server.log"
I see the issue now. I was looking at the concurrent phases of ZGC.
The „Stop the World” phases of ZGC are indeed sub-millisecond
g
Oh, that makes sense