Naor Hadad
03/11/2026, 9:26 AMset-clusters on namespaces, combined with replicateSubscriptionState(true) on the consumer side to sync subscription cursors to the destination. This gets us near-zero downtime: producers keep running, replication backfills the destination, and consumers cut over once backfill is complete.
The problem we keep running into is that replicateSubscriptionState only syncs markDeletePosition - individuallyDeletedMessages does not transfer. As covered in StreamNative article, for subscriptions with any out-of-order acking (which ours have, due to concurrent processing), the redelivery window on the destination can be significant. Our application is not idempotent - double-processing messages causes real issues downstream, so we can't just absorb the duplicates and move on.
Alternative we've considered
Drain the backlog before cutover - stop producers, let consumers fully drain so individuallyDeletedMessages collapses and markDeletePosition catches up, then cut over. This would give us a clean cursor state. The downside is that stopping producers means measurable downtime, and with the volume we're dealing with, the drain window could be long. We'd rather avoid that if there's a better path.
What we're looking for
Has anyone found a clean way to handle this that avoids both the duplicate redelivery problem and extended producer downtime? Open to anything different migration strategy, tooling, a Pulsar feature we've missed, or just a recommendation on how to scope the drain window to make it acceptable. Any experience here would be really helpful.Lari Hotari
03/11/2026, 11:11 AMreplicateSubscriptionState only syncs markDeletePosition - individuallyDeletedMessages does not transfer.
Yes, this is true. However, there has been improvements to the existing solution where it was possible that subscription replication didn't sync the markDeletePosition in all cases. explained in https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1772824988082859
> Our application is not idempotent - double-processing messages causes real issues downstream, so we can't just absorb the duplicates and move on.
This is a problem since Pulsar is at-least-once and there will be duplicates in certain cases in normal operations with a single cluster. Applications should be idempotent or tolerate duplicates in a way or another. Pulsar Transactions doesn't fix this issue since it ties messaging operations to be atomic. If a transaction fails, messages get redelivered, and from the application's perspective they are duplicates.
In Pulsar, it's possible to avoid duplicates on the producer side when a producer retries sending a single message. This feature is disabled by default. If that feature is disabled, producers can produce duplicate messages that are stored in Pulsar.
> Drain the backlog before cutover - stop producers, let consumers fully drain so individuallyDeletedMessages collapses and markDeletePosition catches up, then cut over. This would give us a clean cursor state. The downside is that stopping producers means measurable downtime, and with the volume we're dealing with, the drain window could be long. We'd rather avoid that if there's a better path.
This seems to be a reasonable alternative. There's also a feature in Pulsar that helps reducing the cut off time. It's the "blue-green cluster deployment" (Pulsar 3.0 release blog , PIP-188 document). Unfortunately there's no documentation in the reference doc about this feature (contributions are welcome!)Yuval Levy
03/11/2026, 2:52 PM