https://pulsar.apache.org/ logo
Join Slack
Powered by
# general
  • a

    Andrei

    03/07/2026, 7:50 PM
    Hi guys nice to be here
  • a

    Andrei

    03/07/2026, 7:51 PM
    just wanted to let you know i released this https://github.com/andrei-borcea/quasar-ui it's a dashboard fully functional with all the functionalities for pulsar. as we use pulsar, but the actual ui is not very functional, with this one you can track everything easily.
    👀 1
    pulsarlogo 3
    k
    f
    • 3
    • 13
  • n

    Naor Hadad

    03/11/2026, 9:26 AM
    Hey all looking for some guidance on an EKS-to-EKS Pulsar cluster migration and hoping someone here has been through something similar. Context We're migrating a production Pulsar cluster to a new EKS cluster in the same region. Both clusters run Pulsar 4.0.x deployed via Helm. We have a handful of high-throughput topics with persistent subscriptions that we need to carry over cleanly. Approach we've been evaluating Our primary candidate is geo-replication via
    set-clusters
    on namespaces, combined with
    replicateSubscriptionState(true)
    on the consumer side to sync subscription cursors to the destination. This gets us near-zero downtime: producers keep running, replication backfills the destination, and consumers cut over once backfill is complete. The problem we keep running into is that
    replicateSubscriptionState
    only syncs
    markDeletePosition
    -
    individuallyDeletedMessages
    does not transfer. As covered in StreamNative article, for subscriptions with any out-of-order acking (which ours have, due to concurrent processing), the redelivery window on the destination can be significant. Our application is not idempotent - double-processing messages causes real issues downstream, so we can't just absorb the duplicates and move on. Alternative we've considered Drain the backlog before cutover - stop producers, let consumers fully drain so
    individuallyDeletedMessages
    collapses and
    markDeletePosition
    catches up, then cut over. This would give us a clean cursor state. The downside is that stopping producers means measurable downtime, and with the volume we're dealing with, the drain window could be long. We'd rather avoid that if there's a better path. What we're looking for Has anyone found a clean way to handle this that avoids both the duplicate redelivery problem and extended producer downtime? Open to anything different migration strategy, tooling, a Pulsar feature we've missed, or just a recommendation on how to scope the drain window to make it acceptable. Any experience here would be really helpful.
    🎯 1
    l
    y
    • 3
    • 2
  • a

    Amanda

    03/11/2026, 4:46 PM
    Hi! I am working with a 2 cluster geo-replication set up with a global config store cluster (3 ZKs) all in Kubernetes. I am using Pulsar v4.0.8 and Helm chart v4.4.0. We are heavily testing geo-replication and I am wondering if the following is a bug? 2-way geo-replication has already been configured and enabled at the namespace level between Cluster A and Cluster B • Started a producer only first on Cluster A which auto created the topic under the geo-rep NS. (no consumers started anywhere yet) • Topic was successfully replicated and visible to Cluster B • Producer on Cluster A generated ~100 msgs • Consumer started on Cluster B, starts consuming at msg 101,102,103, so on .. • initial 100 messages sent by producer on cluster A never received by cluster B consumer and 100 messages are sitting in backlog • started consumer on cluster A and it receives all messages starting from message 1 and continues to consume the messages coming in to that topic along with cluster B consumer NOTE: this seems to only be happening when starting the cluster B consumer for the first time on the replicated topic. When the consumer is initially started briefly, then stopped, then started again, it receives all of the messages in the backlog that it missed before starting back up again. Is this a geo-replication bug?
    l
    • 2
    • 6
  • k

    KP

    03/17/2026, 4:18 AM
    Hello hello, got a quick update for Pulsar documentation as skydio continues to expand usage of Pulsar for our message brokering needs 😄 https://github.com/apache/pulsar-site/pull/1093
    🙌 1
    l
    • 2
    • 2
  • f

    Fabri

    03/17/2026, 6:39 PM
    Hello, is there any updates related to the streamnative development for pulsar flink connector? I can see this comment in the last commit ticket sent on november https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLINK-38545 , if streamnative needs more time to include their changes why not generate the new flink connector version for flink 1.20?
  • p

    Partha Talukdar

    03/18/2026, 5:14 PM
    Hey , does the https://pulsar.apache.org/docs/4.1.x/client-libraries-rest/ not work for pulsar v4.0.8. I keep getting
    Copy code
    reason: "Failed to add or retrieve schema."
    👀 1
    a
    • 2
    • 4
  • p

    Partha Talukdar

    03/18/2026, 5:17 PM
    schema validation enforced is set to false, auto update schema is set to true and still not able to get past this
    l
    • 2
    • 1
  • l

    Lari Hotari

    03/23/2026, 7:24 PM
    [VOTE] PIP-460: Scalable Topics (Topics v5) 🚀 Voting thread: https://lists.apache.org/thread/jd9kklo3qlbmt2cxg1s7oynof14gmsn4 PIP-460 introduces a new topic type that uses range-based routing instead of modulo hashing, enabling transparent splitting and merging of streams without breaking key ordering or requiring downtime. This is a parent PIP that establishes the motivation, goals, and key challenges; detailed design for each area (segment model, producer protocol, consumer controller, message entry format, split/merge mechanics, consumer APIs, geo-replication, and more) will follow in dedicated sub-PIPs. Proposal PR: https://github.com/apache/pulsar/pull/25315 Rendered PIP document: https://github.com/lhotari/pulsar/blob/lh-pip-460/pip/pip-460.md Please review the PIP document and vote! (anyone who has joined the dev mailing list can vote)
    👀 1
    🙌 3
  • g

    Grégory (Kosmos)

    03/24/2026, 9:43 AM
    Hi, we're using the Pulsar Client Java. I don't understand the difference between : •
    consumer.negativeAcknowledge(msg);
    •
    consumer.reconsumeLater(msg, 1000, TimeUnit.MILLISECONDS);
    Is it just the possibility to customize the delay ?
    l
    • 2
    • 2
  • a

    Amelia Mink

    03/30/2026, 4:38 AM
    What if the data you don’t have is exactly what your AI needs to succeed? 🔥 Fueling the Future: The Strategic Role of Synthetic Data in AI Workflows In this session, you’ll learn: ✔️ When synthetic data beats traditional augmentation ✔️ How to safely stress-test models without sensitive data risks ✔️ How to prepare for edge cases before they happen ✔️ How to ensure governance, trust, and compliance 🧠 The best AI teams aren’t just building better models—they’re using smarter data strategies. 📅 July 22, 2026 | 2:00 PM ET 📍 Virtual 👉 Register here: https://datasciconnect.com/events/webinars/strategic-role-synthetic-data-ai/ If you want AI that works in production—not just in demos—this is for you. #AI #SyntheticData #MachineLearning #EnterpriseAI #MLOps
  • a

    Amjad Ali

    03/31/2026, 7:18 AM
    Hi, I am using pulsar-all docker image 4.1.2. I am facing two issues related to pulsar. Could anybody have a look at this and let me know whether there is any solution available as of now. https://github.com/apache/pulsar/issues/25433 https://github.com/apache/pulsar/issues/25438
    l
    • 2
    • 2
  • l

    Lari Hotari

    03/31/2026, 11:20 PM
    pulsarlogo 📣 [ANNOUNCE] Apache Pulsar 4.2.0 released 📣 pulsarlogo The Apache Pulsar team is proud to announce Apache Pulsar version 4.2.0. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor management for subscribers, and cross-datacenter replication. For Pulsar release details and downloads, visit: https://pulsar.apache.org/download Release Notes are at: https://pulsar.apache.org/release-notes/versioned/pulsar-4.2.0/ We would like to thank the contributors that made the release possible. Regards, The Pulsar Team
    🙌 5
    👀 3
    🔥 4
    pulsarlogo 3
    k
    • 2
    • 1
  • a

    Amanda

    04/09/2026, 9:25 PM
    Hi! Can anyone provide feedback to the following failed test in our Pulsar environment: Test 1: Pulsar Kubernetes Clusters with Global Configuration Store. Pulsar v4.0.9 Helm Chart v4.5.0 Geo-replication enabled on tenant/ns geo-replication/testing:
    Copy code
    pulsar-talos-toolset-0:/pulsar$ bin/pulsar-admin namespaces get-clusters geo-replication/testing
    pulsar-okd1
    pulsar-talos
    Replicated subscriptions enabled:
    Copy code
    pulsar-talos-toolset-0:/pulsar$ cat conf/broker.conf | grep -i replicatedSu
    enableReplicatedSubscriptions=true
    Configured Auto Failover Client:
    Copy code
    ServiceUrlProvider failover = AutoClusterFailover.builder()
            .primary(primaryUrl)
            //secondary can be a list of multiple back up clusters - good to know for when we deploy a 3+ cluster environment in the future
            .secondary(List.of(secondaryUrl))
            .failoverDelay(30, TimeUnit.SECONDS)
            .switchBackDelay(60, TimeUnit.SECONDS)
            .checkInterval(1000, TimeUnit.MILLISECONDS)
            .build();
    
        PulsarClient pulsarClient = PulsarClient.builder()
            .serviceUrlProvider(failover)
            .build();
    Configure Producer to use failover client and produce messages to a fresh topic (geo-replication/testing/test):
    Copy code
    public SyslogProducer(PulsarClient client) throws PulsarClientException {
            this.client = client;
            this.producer = client.newProducer(Schema.STRING)
                    .topic("persistent://" + CFG.getString("pulsar_tenant") + "/" + CFG.getString("pulsar_namespace") + "/"
                            + CFG.getString("pulsar_topic"))
                    .create();
        }
    producer = new SyslogProducer(new TalosFailoverClient().getAutoFailoverClient());
    Configure Producer to use failover client with replicated subscriptions and consume messages on fresh topic and subscription:
    Copy code
    public SyslogConsumer(PulsarClient client) throws PulsarClientException {
            this.client = client;
            this.consumer = client.newConsumer(Schema.STRING)
                    .topic("persistent://" + CFG.getString("pulsar_tenant") + "/" + CFG.getString("pulsar_namespace") + "/"
                            + CFG.getString("pulsar_topic"))
                    .subscriptionName(CFG.getString("pulsar_subscription"))
                    .replicateSubscriptionState(true)
                    .subscribe();
        }
    consumer = new SyslogConsumer(new TalosFailoverClient().getAutoFailoverClient());
    Tested the following: • started producer on talos cluster on auto created test topic • topic replicated successfully to okd1 cluster (can see it in topics list on okd1 cluster) • started consumer for the first time on talos cluster after 600+ msgs were sent • messages started consuming at message value 660 where did all of the other messages go? • topic stats on talos: 0 msgs in all backlog values replicated connected state = true • topic stats on okd1: backlog size: 120731, replication backlog: 982, replication connected state = false • shutdown/uninstalled talos cluster • producer stopped producing messages with connection error auto cluster failover failed since it did not switch to okd1 cluster and continue producing • started/reinstalled talos cluster: producer never started again Attempting Test 1 again: Using new topic and subscription (still using geo-replication/testing tenant/ns with geo-rep enabled). All same configurations • started producer on talos cluster test1 topic • topic replicated to okd1 successfully • immediately checked topic stats on okd1: okd1 replication connection originally was true with no msgs in the replication backlog or backlog size checked topic stats again shortly after and replication connection switched to false with messages growing in replication backlog and backlog size • checked topic stats on talos: • backlog size is 0 with no consumer even started shouldn’t backlog be growing since no consumer is receiving the msgs? • replication connection = true, backlog size is 0 • started consumer on talos cluster on test 1 topic: • started receiving msgs at msg value 500 continued message loss replication backlog continues to grow on okd1 • stopped consumer on talos test 1 topic, continued producer • started consumer again on talos test 1 topic this time it picked up where it left off - no msg loss • shut down/uninstalled talos cluster: producer and consumer failed to switch to okd1 • restarted/installed talos cluster: producer failed to reconnect and continue
    a
    • 2
    • 1
  • s

    Shasank Pandey

    04/14/2026, 11:03 AM
    Hi, does someone have experience with asynchronous geo replication and cluster failures? I have the following scenario: 1. Cluster A with topic X configured to replicate towards Cluster B. 2. Cluster B with topic X configured to replicate towards Cluster A. Now, lets assume that in a failure scenario, Cluster A goes down. Therefore I have clients producing and consuming only from cluster B. As the cluster A is down, the replicator cannot replicate any messages towards it. Therefore, the replication backlog keeps on increasing, while the topic backlog is near 0, as nearly all messages being produced are also consumed. Theoretically , this scenario can pose a challenge, as it can lead to storage being full, because "acknowledged" messages are still waiting to be replicated. I couldn't find any information about these scenarios in the docs, does anyone have any experience with this?
    l
    • 2
    • 3
  • l

    Lari Hotari

    04/16/2026, 12:04 PM
    [DISCUSS] Address high-severity Jetty 9.4.x CVEs by upgrading to Jetty 12.x in branch-4.0 (breaking changes) https://lists.apache.org/thread/2mcnfn7f0nq3zww3ph2wf41cbh6xcsjj
  • a

    Amanda

    04/16/2026, 5:54 PM
    Can anyone tell me the expected behavior for replication topic stats in a 2 cluster set up with a global metadata store cluster? • manually created topic and subscription using pulsar-admin on cluster A ◦ topic then existed on cluster B because of the global metadata cluster, but the sub did not since that does not belong to global metadata • Run Producer and consumer (using replicated subs) on cluster A - sending and receiving messages successfully on cluster A ◦ view topic stats on cluster A: replication backlog is 0 and replication connected state = true ▪︎ subscription stats show isReplicated false, replicated false ◦ view topic stats on cluster B: replication backlog is growing and replication connected state = false ▪︎ subscription stats show nothing - sub does not exist on cluster B • Stop consumer on cluster A, continue producer on cluster A, start consumer on cluster B, consumer is successfully receiving messages but messages were lost from where consumer A left off and where consumer B picked up ▪︎ view topic stats on cluster A: replication backlog is 0 and replication connected state = true • subscription stats show isReplicated false, replicated false ▪︎ view topic stats on cluster B: replication backlog is growing and replication connected state = false • subscription stats show isReplicated false, replicated false Should the replication state of the remote cluster be false if I am not actively producing messages on that cluster? Am I doing something wrong with replicated subscriptions? Is auto/lazy topic/sub creation causing issues here? Do I need to manually create the topic and sub on both clusters before running producers and consumers? When I manually created the topics and subs, the connected status was true on the remote cluster, and sub replication was working with true stats as well.
    l
    • 2
    • 6
  • j

    João Fernandes

    04/17/2026, 12:36 PM
    Hi everyone, Given the following scenario: 1. Acquire exclusive producer on topic X (single partition) 2. Create reader for topic X 3. Query reader for the topic X latest message ID (e.g.,
    reader.GetLastMessageID()
    in golang) Is it guaranteed that I'll get the latest message ID in that topic? Or will I need to publish a sentinel message and wait to read to guarantee that I'm truly seeing the topic's tail? Thank you!
    l
    • 2
    • 9
  • s

    Shasank Pandey

    04/18/2026, 8:06 AM
    Hi, I was looking into the code for ControlledClusterFailover, and I noticed that all examples use http urls and not https. I dug a bit deeper into the code and I see there is no way to provide the http config to the ControlledClusterFailover implementation as the Builder doesn't expose the option. Personally, i think that we should be providing the option to have https related config for contacting external services. I am willing to also add this as an enhancement but before I do, I wanted to just ask if this is something that would be a good addition?
    d
    • 2
    • 1
  • l

    Lari Hotari

    04/23/2026, 2:58 PM
    Pulsar maintenance releases for supported version series 3.0.x, 4.0.x (current LTS) and 4.2.x are on their way: https://apache-pulsar.slack.com/archives/C5ZSVEN4E/p1776956136214819 A major change that will land in 4.0.10 is the upgrade to Jetty 12.1.8 which is required for CVE fixes.
  • l

    Lari Hotari

    04/28/2026, 2:40 PM
    The Pulsar 4.0.10 release artifacts haven't been successfully synchronized to Maven Central yet. For example pulsar-client is missing. I'll contact ASF infra to fix the problem.
    • 1
    • 1
  • l

    Lari Hotari

    04/29/2026, 2:06 PM
    🔒 Public CA clientAuth EKU sunset — heads up for mTLS users TL;DR: Public CAs are removing clientAuth from TLS leaf certs (rolling out now through 2027). Pulsar deployments using public-CA-issued client certs for mTLS — most notably geo-replication brokers and external producers/consumers — will fail at the next cert renewal under the new policy. Server-only TLS is unaffected. Private PKI (OpenBao, step-ca, Vault, EJBCA, optionally fronted by cert-manager) is the path forward. Full write-up on users@ mailing list: https://lists.apache.org/thread/kbtnz7srvcsg0opj0vo15qm6rcx4btf8 ASF advisory: https://news.apache.org/foundation/entry/the-public-ca-clientauth-eku-sunset-what-apache-software-deployers-need-to-know
  • l

    Lari Hotari

    04/30/2026, 7:22 PM
    📣 pulsarlogo [ANNOUNCE] Apache Pulsar Helm Chart version 4.6.0 Released pulsarlogo 📣 NOTICE: This release splits the ZooKeeper and Broker Kubernetes Services into separate ClusterIP and headless Services. Upgrading from pre-4.6.0 requires StatefulSet re-creation and a brief service disruption is possible. GitOps users (ArgoCD, Flux, Pulumi) should pay special attention. Testing the upgrade in a staging environment is recommended. Please check the release notes for full details before upgrading. --------------- Dear community, The Apache Pulsar team is pleased to announce the release of the Apache Pulsar Helm Chart 4.6.0. The official source release, as well as the binary Helm Chart release, are available at https://www.apache.org/dyn/closer.lua/pulsar/helm-chart/4.6.0/?action=download The helm chart index at https://pulsar.apache.org/charts/ has been updated and the release is also available directly via helm. Release Notes: https://github.com/apache/pulsar-helm-chart/releases/tag/pulsar-4.6.0 Docs: https://github.com/apache/pulsar-helm-chart#readme and https://pulsar.apache.org/docs/helm-overview ArtifactHub: https://artifacthub.io/packages/helm/apache/pulsar/4.6.0 Thanks to all the contributors who made this possible. Regards, The Apache Pulsar Team
    🎉 2
    pulsar party 2
  • s

    Slackbot

    05/08/2026, 9:58 AM
    This message was deleted.
    ✅ 1
    l
    d
    • 3
    • 2
  • u

    萧易客

    05/14/2026, 7:05 AM
    According to https://pulsar.apache.org/contribute/release-policy/#supported-versions @Lari Hotari Hi Lari, is 3.0.17 the final release of the 3.0 LTS version? Is there a chance to extend the support period of the 3.0 LTS version branch?
    l
    • 2
    • 1
  • a

    Amanda

    05/21/2026, 7:13 PM
    Hello! I am experimenting with auto cluster failover and have it configured in my Pulsar client code that I am using to test with 2 Kubernetes clusters. From what we can tell, it is working. Although, we would like to be able to verify this in the logs - especially for when we bring cluster1 back up after failing, we'd like to know that our producer and consumer actually switched back. I've been looking deep into the broker pod logs but can't find anything verifying the failover or switch back. Is there a way we can verify?
    l
    • 2
    • 1
  • d

    Daniel Kaminski

    05/26/2026, 2:32 PM
    Hi, we are using the Pulsar version 4.0.10 and would like to stay on the LTS Versions. Do you have any specific plans to release version 5.0.x? I am asking as customers are interested in potentially leveraging SMT functionality for unwrapping messages when using CDC. There is support for that in 4.1.x but as we would like to stay on the LTS Versions the question is when 5.0.x would be released.
    l
    • 2
    • 2
  • f

    Fabri

    05/27/2026, 6:05 AM
    Hello, I noticed there was an intention to release a new version of the Flink connector for Pulsar (v4.2.0): https://lists.apache.org/thread/bwd91nbcyflbhvhrm33qg7tjksl6b03w However, no binding votes were cast to approve or reject the request — only non-binding votes were submitted. What would be the next steps here? Thanks!
  • g

    Grégory (Kosmos)

    05/28/2026, 5:07 PM
    Hi, I'm stuck because I don't understand how to consume DLQ message (or creation of DLQ topic). I ran pulsar with this command :
    Copy code
    docker run -it -p 6650:6650 -p 8080:8080 -e TZ=Europe/Paris --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:3.0.17 bin/pulsar standalone --advertised-address localhost
    Then I have these 3 java files, I run SimpleConsumerWithRetry, and gets logs :
    Copy code
    >> pulsar client created
    >> pulsar DLQ consumer created
    >> pulsar consumer created
    Then I run SimpleProducer, here the logs
    Copy code
    >> pulsar client created
    >> pulsar producer created
    deliver msg 9:1:-1, value:Message de 2026-05-28T19:00:43.731+02
    Then I look at the log of SimpleConsumerWithRetry :
    Copy code
    2026-05-28T19:00:43.798+02: receive msg 9:1:-1, value:Message de 2026-05-28T19:00:43.731+02
    2026-05-28T19:00:45.902+02: receive msg 18:0:-1, value:Message de 2026-05-28T19:00:43.731+02
    2026-05-28T19:00:48.899+02: receive msg 18:1:-1, value:Message de 2026-05-28T19:00:43.731+02
    So far so good : 2 retries, with 2 seconds delay. But the message is not send to the DLQ (at least, not the DLQ topic I'm listening). If I ask the topic list
    curl --silent -XGET <http://localhost:8080/admin/v2/persistent/public/testns>
    there is only 2 topics (I was expected 3 : main, retry, dlq) :
    ["<persistent://public/testns/test>","<persistent://public/testns/test-subs-DLQ>"]
    . I tried also to not listen to test-subs-DLQ (removing one consumer in the code) and there is only 1 topic created (test). Any clue ?
    SimpleProducer.javaConstants.javaSimpleConsumerWithRetry.java
    a
    • 2
    • 2
  • g

    Grégory (Kosmos)

    05/29/2026, 3:42 PM
    Hi everyone, I noticed something unusual with
    reconsumeLater
    . When I set
    delayTime
    to around 1 second (or less), the message is redelivered immediately. I wrote a small integration test to reproduce this behavior (the test requires a valid Docker installation to start a Pulsar container). When running the
    TestReconsumeLater.java
    test, if the constant
    REDELIVER_DELAY
    is set to less than 1010 ms, the test fails because the message is redelivered in 34 ms instead of the expected delay:
    Copy code
    java.lang.AssertionError:
    Expected actual:
      34L
    to be greater than or equal to:
      1009L
    Is this a known limitation ? Is it documented anywhere ?
    pulsar-reconsume-later.zip
    d
    • 2
    • 3
1...151152153154155Latest