Suhas Karanth
03/26/2025, 9:49 AMRunLLM
03/26/2025, 9:49 AMConsumerOptions to enable message ordering. However, message ordering in Pulsar is typically managed through the use of keys. When using a Key_Shared subscription type, messages with the same key are delivered to the same consumer, maintaining order for those keys.
To achieve message ordering, you can ensure that your producer assigns keys to messages. The consumer should then use a Key_Shared subscription type to maintain order for messages with the same key. You can refer to the Pulsar documentation on Key_Shared subscriptions for more details.
Would you like more detailed guidance on setting up a producer and consumer with key-based ordering?
Sources:
- https://github.com/apache/pulsar-client-node/issues/339
- https://pulsar.apache.org/docs/3.2.x/concepts-messaging#partitioned-topics 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.Suhas Karanth
03/26/2025, 9:53 AMRunLLM
03/26/2025, 9:53 AMKey_Shared subscription type for your consumer. This ensures that messages with the same key are delivered to the same consumer, maintaining order for those keys.
Here's a step-by-step guide with code examples:
Producer Code Example
package main
import (
"context"
"fmt"
"log"
"github.com/apache/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "my-topic",
})
if err != nil {
log.Fatal(err)
}
defer producer.Close()
ctx := context.Background()
for i := 0; i < 10; i++ {
_, err := producer.Send(ctx, &pulsar.ProducerMessage{
Key: fmt.Sprintf("key-%d", i%3), // Assign a key to each message
Payload: []byte(fmt.Sprintf("message-%d", i)),
})
if err != nil {
log.Fatal(err)
}
}
}
Consumer Code Example
package main
import (
"context"
"fmt"
"log"
"github.com/apache/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
consumer, err := client.Subscribe(pulsar.ConsumerOptions{
Topic: "my-topic",
SubscriptionName: "my-subscription",
Type: pulsar.KeyShared, // Use Key_Shared subscription type
})
if err != nil {
log.Fatal(err)
}
defer consumer.Close()
for {
msg, err := consumer.Receive(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Printf("Received message msgId: %v -- content: '%s'\n", msg.ID(), string(msg.Payload()))
consumer.Ack(msg)
}
}
Key Points
- Key Assignment: The producer assigns a key to each message. Messages with the same key will be delivered to the same consumer.
- Key_Shared Subscription: The consumer uses the Key_Shared subscription type to ensure that messages with the same key are processed by the same consumer, maintaining order.
This setup ensures that messages with the same key are processed in order by the same consumer. If you have further questions or need additional assistance, feel free to ask!
Sources:
- https://github.com/apache/pulsar-client-go/blob/master/pulsar/consumer_test.go
- https://pulsar.apache.org/docs/3.2.x/concepts-messaging#sticky 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.Suhas Karanth
03/26/2025, 9:55 AMRunLLM
03/26/2025, 9:55 AMKey or OrderingKey to ensure message ordering, particularly in a Key_Shared subscription. The OrderingKey is specifically used to dictate the order of message dispatch within the Key_Shared mode, while the Key is used for routing messages to specific consumers.
If you specify the same value for both Key and OrderingKey, it should not have a negative impact. Using the same value ensures that messages are routed and ordered consistently based on the same criteria. This can be beneficial if you want to ensure that messages with the same key are always processed in order by the same consumer.
Would you like more detailed information on how these keys are used in practice?
Sources:
- https://github.com/apache/pulsar/issues/23212 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.Suhas Karanth
04/03/2025, 10:05 AMConsumerOptions struct in pulsar Go client library, there is both SubscriptionName and Name. What is the purpose of both? What purpose do they serve?RunLLM
04/03/2025, 10:05 AMConsumerOptions struct includes both SubscriptionName and Name fields, each serving distinct purposes:
1. SubscriptionName: This is a required field that specifies the name of the subscription for the consumer. It is used to identify the subscription on the broker and is essential for managing message delivery and acknowledgment. The subscription name allows multiple consumers to share the same subscription, enabling different subscription modes like Exclusive, Shared, Failover, and Key_Shared.
2. Name: This field is used to assign a custom name to the consumer instance itself. It is optional and primarily serves for identification and management purposes, such as logging and monitoring. If not specified, a default consumer name is generated. The consumer name does not affect the subscription or message delivery but can be useful for tracking and debugging.
Would you like more detailed information on how these fields are used in practice?
Sources:
- ConsumerOptions in Pulsar Go client 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.Suhas Karanth
04/08/2025, 7:49 AMRunLLM
04/08/2025, 7:49 AMName field in the ConsumerOptions is optional and does not affect the subscription or message delivery, using unique consumer names can be beneficial for tracking, logging, and monitoring purposes. It helps in identifying and managing individual consumers, especially in scenarios where you need to debug or analyze consumer behavior.
Using the same name for multiple consumers is technically possible, but it may lead to confusion in logs and monitoring tools, as it becomes challenging to distinguish between different consumer instances. Therefore, assigning unique names to each consumer is a best practice.
Would you like more detailed guidance on setting up consumers with unique names?
Sources:
- ConsumerOptions in Pulsar Go client 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.