@RunLLM What is the combination for producer parameter to be used for blockifqueue full and producer exception for the backlog quota

Set the producer parameter

blockIfQueueFull=true

to make the producer block when its outgoing message queue is full. For backlog quota, set the retention policy to

producer_exception

so the broker throws an exception when the backlog limit is exceeded. This combination ensures the producer blocks on local queue full, but fails with an exception if the broker backlog quota is hit. For example, in Java:

Producer<byte[]> producer = client.newProducer()
    .topic("my-topic")
    .blockIfQueueFull(true)
    .create();

Set the backlog quota policy using Pulsar admin:

pulsar-admin namespaces set-backlog-quota my-tenant/my-ns \
  --limit 10G \
  --policy producer_exception

According to ProducerBuilder.java and Pulsar backlog quota docs, this is the recommended configuration. Would you like more detail? Sources: - Pulsar backlog quota docs - ProducerBuilder.java 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@RunLLM But the producer is hung in the reative streams when using this producer_exception and block if queue full set to true .

If you set

blockIfQueueFull=true

and the backlog quota policy to

producer_exception

, the producer will block when its local queue is full, but if the broker backlog quota is exceeded, the broker will throw a

ProducerBlockedQuotaExceededException

and fail all pending messages. In Pulsar's Java client, the async API (used by reactive streams) can still block threads if

blockIfQueueFull=true

, which is problematic for non-blocking/reactive applications. This is a known limitation and is discussed in the Pulsar community: the async API does not provide a non-blocking backpressure mechanism when the queue is full, leading to potential hangs in reactive streams scenarios (PIP-385 discussion). To avoid blocking in reactive streams, set

blockIfQueueFull=false

so the producer fails fast with

ProducerQueueIsFullError

instead of blocking. This allows your application to handle backpressure in a non-blocking way (ProducerBuilder.java). Would you like more detail? Sources: - PIP-385 Add rate limit semantics to pulsar protocol and Java Client - ProducerBuilder.java 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.