<@U078Y59KC3B> in Key_Shared with a partitioned to...
# ask-ai
j
@RunLLM in Key_Shared with a partitioned topic. Do we have a garauntee if the a particular key will only go to one particular "partition". I am already aware of this garauntee at a consumer lebel. This would involve the hash range check would happen before the message is sent to the broker. is this how it is implemented?
r
No, in Pulsar's Key_Shared subscription with partitioned topics, there is no guarantee that a particular key will always be routed to the same partition. The assignment of a key to a partition is determined by the producer's routing logic, which typically hashes the key and maps it to a partition, but this is not a strict guarantee at the broker level. The hash range check and key-to-partition mapping are performed on the producer side, not the broker, using the configured routing mode and hashing scheme. The broker only guarantees that, within a partition, all messages with the same key are delivered to the same consumer in order, but does not enforce key-to-partition affinity across the entire topic. According to Pulsar documentation and Message RoutingMode, the producer hashes the key and selects a partition, but this mapping can change if the number of partitions changes or if the producer's routing configuration changes. The broker does not perform a hash range check before accepting the message; it simply receives messages on the partition as determined by the producer. Would you like more detail on how the producer's routing logic works? Sources: - Pulsar: Partitioned Topics - Pulsar: Message RoutingMode (Java) 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.
j
@RunLLM Got it. Please validate this for me: the "partitionKey" that is provided when emitting a message, is what is used when selecting the partition that this message would be routed to. the "orderingKey" is what is used in key shared subscription mode to select the consumer. when not provided, partitionKey is used for the same purpose. This would mean, if the same key is used as the partition key, and is also used as the ordering key, then a particular key, would always go through one partition, and to one consumer Correct?
r
Partially correct. The "partitionKey" (or "key") is used by the producer to select the partition for the message—this determines which partition the message is routed to. The "orderingKey" is used in Key_Shared subscription mode to select the consumer within that partition; if "orderingKey" is not set, the "partitionKey" is used for consumer assignment as well. However, even if you use the same value for both "partitionKey" and "orderingKey," there is no strict guarantee that a key will always go to the same partition if the number of partitions changes or the routing logic changes. But as long as the partition count and routing logic remain unchanged, a given key will always be routed to the same partition and, within that partition, to the same consumer. See Partitioned Topics and Key_Shared subscription. Would you like more detail on edge cases or partition changes? Sources: - Pulsar: Partitioned Topics 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.