Folks, I have been looking into

ThresholdShedder

and I see in tag 3.0.2 that

updateAvgResourceUsage

method does not take into account

loadBalancerMemoryResourceWeight

but only

loadBalancerDirectMemoryResourceWeight

. Is this intentional? Why exclude memory usage from load shedding logic?

there's also another explanation. I'll lookup the reference.

Thanks a lot @Lari Hotari! I understand the wider context now. I plan to stick to the LTS release. Would it make sense for me to set the direct memory weight to 0 as well? One of the primary issues that we are facing with our current setup of 2.7.x is that there are frequent load shedding happening due to memory usage breaches. During this we also note that there are seeing some errors in publishing messages into pulsar topics.

Would it make sense for me to set the direct memory weight to 0 as well? One of the primary issues that we are facing with our current setup of 2.7.x is that there are frequent load shedding happening due to memory usage breaches. During this we also note that there are seeing some errors in publishing messages into pulsar topics.

@Daniel I think experimentation and testing is a good way to verify if a specific configuration change works for you. For memory usage in the broker, there are at least 2 changes that address a lot of problems. You will have to specifically enable the feature by setting managedLedgerMaxReadsInFlightSizeInMB to a value in the range, let say 100 to 400 (depending on your RAM size, this uses direct memory). Changes were https://github.com/apache/pulsar/pull/18245 and https://github.com/apache/pulsar/pull/17241 that make the difference in high fanout catch up read scenarios which don't behave nicely without those improvements. In addition, I'd set maxMessagePublishBufferSizeInMB to a fixed value instead of relying on the default value that is 50%. A good value could be 500. The broker cache needs also tuning, but it's hard to do general recommendations. I have a presentation made in 2022 about something that helps in understanding how to do tuning: https://www.apachecon.com/acna2022/slides/03_Hotari_Lari_Performance_tuning_Pulsar.pdf . Recording is . It's not a cookbook but rather "teaching to fish". 🙂

Thank you, @Lari Hotari. We are actively looking to upgrade our Pulsar cluster 🙂. Thanks for the heads up on the GC. I will make changes to our upgrade plan and also look into these resources.

Thread about ZGC https://apache-pulsar.slack.com/archives/C5ZSVEN4E/p1708692059819849?thread_ts=1708674559.930179&cid=C5ZSVEN4E