@RunLLM with a keyshared subscription for topic A, i'm using messages with the same partition key showing up in multiple consumers, how is that possible?

If messages with the same partition key are being delivered to multiple consumers in a Key_Shared subscription, the most likely cause is that your consumers are not using the same subscription name. In Pulsar, each unique subscription name creates a separate subscription, so each will receive all messages independently, regardless of key. For Key_Shared to work as intended (messages with the same key go to the same consumer), all consumers must use the same subscription name and Key_Shared type. This is a common mistake and is confirmed in this issue and this issue. Additionally, ensure that your producer is correctly setting the key (or orderingKey/partitionKey, depending on client language), and that batching is either disabled or set to key-based batching, as default batching can break Key_Shared guarantees (Pulsar docs, batching details). Would you like more detail or troubleshooting steps? Sources: - KeyShared Consumer Issue Example - KeyShared Subscription Type - Batching for KeyShared Subscriptions - KeyShared Consumer Example 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.

Why does default batching break key_shared guarantees?

Default batching breaks Key_Shared guarantees because it can mix messages with different keys into the same batch. When the broker dispatches a batch, it assigns the entire batch to a consumer based on the key of the first message, causing messages with the same key to be delivered out of order or to different consumers. This violates the Key_Shared guarantee that all messages with the same key are delivered to the same consumer in order. Key-based batching solves this by ensuring each batch contains only messages with the same key (Pulsar docs). Would you like a step-by-step technical explanation? Sources: - Batching for Key_Shared Subscriptions 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.

@RunLLM does keybased batching support other types of subscriptions?

Yes, key-based batching can be used with other subscription types (Exclusive, Failover, Shared), but it is specifically required for Key_Shared to preserve ordering guarantees. For other subscription types, default batching is sufficient and key-based batching is not necessary, though it will still function (Pulsar docs). Would you like more detail on how batching interacts with each subscription type? Sources: - Batching for Key_Shared Subscriptions 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.

@RunLLM does topic partitioning break key shared guarantees as well?

No, topic partitioning does not break Key_Shared guarantees. Within each partition, Key_Shared ensures that all messages with the same key are delivered to the same consumer in order. However, if the same key appears in different partitions, ordering is only guaranteed per partition, not across partitions (Pulsar docs, Key_Shared subscription). Would you like more detail on how partitioning and Key_Shared interact? Sources: - Partitioned Topics - Key_Shared subscription 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.