<@U078Y59KC3B> under what circumstances should I c...
# ask-ai
s
@RunLLM under what circumstances should I configure
defaultNamespaceBundleSplitAlgorithm
to something other than the default?
r
You should configure
defaultNamespaceBundleSplitAlgorithm
to something other than the default (
range_equally_divide
) if your use case requires bundle splits based on topic count, specific hash positions, or message rate/throughput, rather than just evenly dividing the hash range. For example, use
topic_count_equally_divide
to balance the number of topics per bundle,
specified_positions_divide
for custom split points, or
flow_or_qps_equally_divide
to split based on message rate or throughput thresholds. This is useful for optimizing load distribution in clusters with uneven topic loads or specific performance requirements, as described in ServiceConfiguration.java and NamespaceBundleSplitAlgorithm.java. Would you like more detail on when to use each algorithm? Sources: - ServiceConfiguration.java - NamespaceBundleSplitAlgorithm.java 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM what are the possible conditions under which there would be uneven topic loads?
r
Uneven topic loads can occur when some topics receive significantly more traffic (messages or throughput) than others, or when a few topics have many producers/consumers while others have few. This can result from application usage patterns, hot topics, or imbalanced partitioning, leading to certain bundles or brokers becoming hotspots while others remain underutilized, as explained in Broker load balancing Concepts. Would you like more detail or examples of these scenarios? Sources: - Broker load balancing Concepts 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM is the broker load balancer a separate component or is it something that runs on each broker and is responsible for redirecting the request?
r
The broker load balancer is not a separate component; it runs on each broker and is responsible for balancing topic sessions across brokers based on dynamic load data. Any broker can respond to client lookup requests and redirect them to the appropriate broker as needed, as described in Broker load balancing Concepts and StreamNative Broker Load Balancer. Would you like more detail on how the load balancer operates? Sources: - Broker load balancing Concepts - StreamNative Broker Load Balancer 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM what is the recommended value for
loadBalancerNamespaceBundleMaxMsgRate
?
r
There is no universally recommended value for `loadBalancerNamespaceBundleMaxMsgRate`; it should be set based on your cluster's expected message rate and performance characteristics. The default value is 30,000 messages per second, but you may need to adjust it higher or lower depending on your workload and when you want bundle splits to be triggered, as shown in ServiceConfiguration.java and Pulsar Load Balance Docs. Would you like guidance on how to determine the best value for your use case? Sources: - ServiceConfiguration.java - Pulsar Load Balance Docs 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM what is the configuration option for setting bundle unloading strategy?
r
The configuration option for setting the bundle unloading strategy is
loadBalancerLoadSheddingStrategy
in the
broker.conf
file. You can set it to strategies like
ThresholdShedder
,
OverloadShedder
, or
UniformLoadShedder
depending on your needs, as described in Pulsar Load Balance Docs and Broker Load Balancing Concepts. Would you like more detail on each available strategy? Sources: - Pulsar Load Balance Docs - Broker Load Balancing Concepts 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM @RunLLM under what circumstances should I configure
loadBalancerLoadSheddingStrategy
to something other than the default?
r
You should configure
loadBalancerLoadSheddingStrategy
to something other than the default (
ThresholdShedder
) if your cluster has specific load balancing needs, such as requiring immediate unloading when a broker exceeds a resource threshold (
OverloadShedder
), or if you want to distribute load uniformly by comparing the most and least loaded brokers (
UniformLoadShedder
). Each strategy is optimized for different scenarios, as detailed in Broker Load Balancing Concepts. Would you like more detail on which strategy fits which scenario? Sources: - Broker Load Balancing Concepts 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
s
@RunLLM should the configuration for
defaultNumberOfNamespaceBundles
be dependent in any way on the number of broker nodes that will be present in the cluster?
r
Yes, the configuration for
defaultNumberOfNamespaceBundles
should generally be set higher than the number of broker nodes in the cluster to ensure even distribution of load. For example, if you have 16 brokers, using 64 bundles achieves good distribution, as recommended in Pulsar Load Balance Docs. Would you like more detail on how to choose the optimal number of bundles? Sources: - Pulsar Load Balance Docs 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.