hi Everyone, I am trying to use multitopic consume...
# general
d
hi Everyone, I am trying to use multitopic consumer and setting receiverQueueSize as 1 (as this queue size should be >0). then after some time I am creating one more new consumer with the newly added topic from different tenant, and started receiving message from newly created consumer . Below are the observations , need to know weather this is valid behavior in pulsar. 1. With the new consumer messages are receiving properly for newly added topic. 2. but for old consumer we have 3 messages left , and once we close the old consumer , these 3 messages started consuming in new consumer with the same message id with the redelivery count as 1. 3. Tried with new pulsar client and new consumer name , behavior is same.
Copy code
allTopicsConsumerOld = pulsarClient.newConsumer()
        //.topic("pulsartenant_1/default/test-partition-0") // this is for Single consumer
        .topic("pulsartenant_1/default/test") // this is for multiconsumer
        .consumerName("my-consumer")
        .subscriptionName("my-sub")
        .subscriptionType(SubscriptionType.Shared)
        .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
        .receiverQueueSize(1)
        .subscribe();
Copy code
allTopicsConsumerNew = pulsarClient.newConsumer()
        //.topic("pulsartenant_1/default/test-partition-0") // this is for Single consumer
        .topic("pulsartenant_1/default/test","pulsartenant_2/default/test") // this is for multiconsumer
        .consumerName("my-consumer")
        .subscriptionName("my-sub")
        .subscriptionType(SubscriptionType.Shared)
        .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
        .receiverQueueSize(1)
        .subscribe();
l
redelivery count is kept in memory and won't be preserved after the topic is "unloaded". That's something that should be clarified in the documentation.
d
Thanks for replying .... Saw InMemoryRedeliveryTracker.java file , here redelivery count mentioned in-memory ... but we are not doing any unload explicitly while creating new consumer ( will it internally getting unloaded while creation of new consumer?) and new consumer created with the same topic name with addition of new topic. But I am seeing only 3 messages are getting with redelivery count is 1. Is this the correct behavior?
l
Unloading can happen at any time, triggered by the Pulsar load balancer.
d
Thanks a lot @Lari Hotari I am seeing receiverQ size should be > 0 incase of multitopic consumer , any reason behind this ? Whereas for single topic consumer we can set receiveQ size as 0.
l
I am seeing receiverQ size should be > 0 incase of multitopic consumer , any reason behind this ?
Well it would be pointless to have a messaging solution that doesn't have any messages inflight. If you are having trouble with it, it's possible that you are trying to solve a problem in a way which could be in conflict with Pulsar's architecture. What is the problem you are trying to solve and what is the use case? There are usually multiple ways and other ways could be a better fit for Pulsar.
d
Thanks @Lari Hotari I am wandering how pulsar allowed receiveQ size as 0 for a single topic consumer . We have the following scenario. Assume that we have 5 topics. We want to have a single Shared Subscription consumer across the 5 topics to consume the message in round robin fashion across the 5 topics. We do not want the consumer to hold the message in receive queue. The consumer needs to pick the message only it intends to handle and handle over the message to some other process. In my case, the number of topics 5 is not constant. It can keep changing. To handle the above situation with multi-topic consumer, when a new topic gets added, I am trying to close the existing consumer and create a new consumer with additional topics. Also, please note that the topics will be across the pulsar tenants (i.e, the topics will not be from same single pulsar tenant).