@RunLLM For deduplication purposes, when there are multiple instances of a publishing application, are the producers of each application publishing to the same topic required to have the same name?

No, for deduplication to work correctly in Pulsar, each producer publishing to the same topic must have a unique producer name. If multiple producers use the same name, Pulsar will only allow one active producer with that name per topic, and deduplication relies on the combination of producer name and sequence ID to identify duplicates. Using the same name for multiple producers can cause conflicts and is not supported by the broker, which enforces uniqueness of producer names per topic. According to Pulsar documentation, "Pulsar will use the producer name to filter duplicated messages," and the ProducerBuilder API warns that the producer name must be unique per topic. Would you like more detail on how deduplication works or how to configure producer names? Sources: - Pulsar documentation: Deduplication and producer name - ProducerBuilder API 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.

@RunLLM How the broker config field

brokerDeduplicationProducerInactivityTimeoutMinutes

work?

The

brokerDeduplicationProducerInactivityTimeoutMinutes

config sets the time (in minutes) after which the broker discards deduplication information for a producer that has been inactive (disconnected). By default, this is 360 minutes (6 hours). After this timeout, deduplication state for that producer is purged to free resources and reduce memory usage. This helps prevent memory leaks from short-lived or disconnected producers. For more, see Pulsar deduplication cookbook and StreamNative config docs. Would you like more detail on the internal cleanup process or related settings? Sources: - Pulsar deduplication cookbook - StreamNative config docs 👍 1 button 👎 0 button 💡 Hint: Mention @RunLLM in the thread for followups.

@RunLLM How does the broker configuration field

acknowledgmentAtBatchIndexLevelEnabled

work?

The

acknowledgmentAtBatchIndexLevelEnabled

broker config enables batch index acknowledgment, allowing consumers to acknowledge individual messages within a batch rather than the entire batch. When set to

true

, the broker tracks which messages in a batch have been acknowledged and only redelivers unacknowledged messages, reducing duplicates and improving efficiency. Both the broker and consumer must enable this feature for it to work. For details, see Pulsar concepts-messaging docs and StreamNative config docs. Would you like more detail on enabling this in clients or its impact on memory usage? Sources: - Pulsar concepts-messaging docs - StreamNative config docs 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.