<@U078Y59KC3B> I have pulsar installed via the hel...
# ask-ai
a
@RunLLM I have pulsar installed via the helm. I currently have a Kubernetes service deployed with a replica set of 16. I produce a message that creates a backlog of 16 messages in a topic but for whatever reason, when my service (which is the consumer) goes to work, only 8 of the 16 pods are working on the pulsar topic. Any ideas as to why all 16 aren't working to burn down the backlog?
r
The issue you're experiencing could be related to the subscription type and the way messages are distributed among consumers in Apache Pulsar. In Pulsar, the subscription type determines how messages are distributed to consumers. If you're using a
Shared
subscription, messages are distributed among all consumers in a round-robin fashion, which should allow all 16 pods to consume messages. However, if you're using an
Exclusive
or
Failover
subscription, only one consumer (or a subset in the case of failover) will receive messages at a time. Another possibility is that the receiver queue size or the consumer configuration might be limiting the number of messages each consumer can process at a time. According to Pulsar documentation, the receiver queue size determines how many messages a consumer can fetch at a time. If this is set too low, it might limit the throughput of each consumer. Would you like more detailed guidance on how to configure your consumers or check the subscription type? Sources: - Pulsar documentation on message queues 👍 0 button 👎 1 button 💡 Hint: Mention @RunLLM in the thread for followups.