This message was deleted pulsar #general

Join Slack

This message was deleted.

# general

Slackbot

02/12/2024, 5:17 PM

This message was deleted.

Jerome B

02/12/2024, 5:18 PM

Here it is how consumer are created

Copy code

for (int i = 0; i < concurrentConsumer; i++) {

  ConsumerBuilder<GenericRecord> consumerBuilder = client.newConsumer(myschema)
         .topic("mytopic")
         .consumerName(hostname+"-"+i)
         .subscriptionName("mysubcription")
         .subscriptionMode(SubscriptionMode.Durable)
         .subscriptionType(SubscriptionType.Key_Shared)
         .keySharedPolicy(KeySharedPolicy.autoSplitHashRange().setAllowOutOfOrderDelivery(true))
         .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
         .receiverQueueSize(1000)
         .maxAcknowledgmentGroupSize(100)
         .acknowledgmentGroupTime(100, TimeUnit.MILLISECONDS)
         .messageListener((MessageListener<GenericRecord>) (consumer, msg) -> {
            // my consumer code
          }
}

David K

02/12/2024, 5:42 PM

Is this behavior limited to just the one hash range?

Jerome B

02/12/2024, 5:49 PM

no, I found several hash ranges duplicated on two consumers, it's also not linked to the host.

David K

02/12/2024, 5:58 PM

Thanks. So you have 4x40 consumers on the same partitioned topic. How many partitions are on the topic?

Jerome B

02/12/2024, 6:09 PM

We have got 9 partitions on the topic.

Lari Hotari

02/12/2024, 6:21 PM

there might be an explanation for this. By default org.apache.pulsar.broker.service.ConsistentHashingStickyKeyConsumerSelector is used when

KeySharedPolicy.autoSplitHashRange()

is selected. https://github.com/apache/pulsar/issues/13305 for more details. It seems that hash ranges aren't unique in the internal data structure. That's why the keyHashRanges information might not be relevant.

Jerome B

02/12/2024, 6:34 PM

ConsistentHashing is enabled

Copy code

# On KeyShared subscriptions, with default AUTO_SPLIT mode, use splitting ranges or
# consistent hashing to reassign keys to new consumers
subscriptionKeySharedUseConsistentHashing=true

# On KeyShared subscriptions, number of points in the consistent-hashing ring.
# The higher the number, the more equal the assignment of keys to consumers
subscriptionKeySharedConsistentHashingReplicaPoints=100

Sorry to forgot mention that

Jerome B

02/12/2024, 6:36 PM

(I'll try tomorrow disabling that parameter)

🤞 1

Lari Hotari

02/12/2024, 6:42 PM

Are you running into an actual problem? The duplicate hash ranges in the stats don't seem to be a sign of a problem with ConsistentHashingStickyKeyConsumerSelector. The implementation addresses clear problems and is therefore preferred. https://github.com/apache/pulsar/pull/6791 explains the background.

Lari Hotari

02/12/2024, 6:49 PM

if I look keyHashRanges field, I found that the following range
"[1932476396, 1932750362]"
is associated to two different consumers
host2-2
and
host2-25

That is because of the weakness of the current way the internal hashes are created in ConsistentHashingStickyKeyConsumerSelector: https://github.com/apache/pulsar/blob/08b28f5c5cbbdc63467455c40106173475ff01e5/pul[…]/broker/service/ConsistentHashingStickyKeyConsumerSelector.java

Lari Hotari

02/12/2024, 6:50 PM

host2-2

and

host2-25

would collide for example with key

host2-251

. That would be i=51 for

host2-2

and i=1 for

host2-25

. The workaround would be to add some other character at the end of the consumer name.

Lari Hotari

02/12/2024, 6:51 PM

simple workaround:

Copy code

.consumerName(hostname+"-"+i+"-")

➕ 1

👀 1

Lari Hotari

02/13/2024, 11:36 AM

@Jerome B I think it's worth an issue report in https://github.com/apache/pulsar/issues about the hash collision that happens with your use case where you have a consumerName ending with a number. (

host2-2

and

host2-25

consumer names collide).

👍 1

Lari Hotari

02/13/2024, 11:37 AM

The problem doesn't seem to be a severe issue, but it's worth reporting so that we could address it. The workaround is to add some character other than a number to the end of the consumer name.

Jerome B

02/13/2024, 2:33 PM

@Lari Hotari I just tested your workaround and it fix my issue thanks a lot. issue is created : https://github.com/apache/pulsar/issues/22050

Lari Hotari

02/13/2024, 2:49 PM

thanks!

Lari Hotari

02/14/2024, 6:05 PM

created https://github.com/apache/pulsar/pull/22053 to fix it

Jerome B

02/15/2024, 9:32 AM

Excellent, thanks very much, in the next version, we'll get back our consumer name without "-" at the end 🙂

4 Views

Open in Slack

Previous Next