Hi everyone, Given the following scenario: 1. Ac...
# general
j
Hi everyone, Given the following scenario: 1. Acquire exclusive producer on topic X (single partition) 2. Create reader for topic X 3. Query reader for the topic X latest message ID (e.g.,
reader.GetLastMessageID()
in golang) Is it guaranteed that I'll get the latest message ID in that topic? Or will I need to publish a sentinel message and wait to read to guarantee that I'm truly seeing the topic's tail? Thank you!
l
Short answer: yes, it's guaranteed — no sentinel message needed. Given your setup (non-partitioned topic, exclusive producer, non-transactional messages), after
producer.Send()
returns successfully for message M, a subsequent
reader.GetLastMessageID()
call is guaranteed to return a message ID ≥ M. Why — broker-side ordering (apache/pulsar) The key invariant lives in
PersistentTopic.addComplete()
, the BookKeeper write-ack callback. Inside that callback the order is: 1. BookKeeper acks the write →
ManagedLedgerImpl
advances its
lastConfirmedEntry
(LAC). This happens in the managed-ledger machinery before
addComplete
fires. 2.
addComplete
runs on the managed-ledger thread and reads
ledger.getLastConfirmedEntry()
(syncing it into the transaction buffer's max-read-position). 3. Then
publishContext.completed()
is called, which schedules the
SendReceipt
onto the
ServerCnx
I/O thread. 4. The client receives the receipt and completes the send future. So by the time your producer's
Send()
returns,
getLastConfirmedEntry()
has already advanced past M on that broker. There is no window where the ack has been sent but the LAC hasn't moved. The GetLastMessageId path
ServerCnx.handleGetLastMessageId()
calls
PersistentTopic.getLastDispatchablePosition()
. For a non-transactional topic this returns
ledger.getLastConfirmedEntry()
directly — the same field that was updated before your ack. With transactions disabled there's no "max read position" indirection to worry about. Go client (apache/pulsar-client-go)
Reader.GetLastMessageID()
is not cached — it always round-trips to the broker. The call chain is:
reader.GetLastMessageID
→
partitionConsumer.getLastMessageID
→
getLastMessageIDAndMarkDeletePosition
→
eventsCh
→
internalGetLastMessageID
→
requestGetLastMessageID
which sends a
CommandGetLastMessageId
via
rpcClient.RequestOnCnx()
and blocks for the response. Fresh view from the broker on every call. Why the exclusive producer matters Exclusive producer fencing doesn't change the ack-ordering above — that applies to any producer. What it buys you is that no other writer can slip a message in between your publish and your query. So the "latest message ID" you see is genuinely yours, not racing with a concurrent publisher. Caveats worth knowing • Assumes same broker / same ManagedLedger instance — i.e., no topic unload or bundle transfer between publish and query. On unload, a new broker recovers LAC from the BookKeeper ensemble, so the invariant still holds, but your query will block until load completes. • On partitioned topics,
GetLastMessageID
on a multi-topic reader isn't supported in the Go client (returns an error); query each partition's reader individually. • If transactions are ever enabled,
getLastDispatchablePosition
switches to
TransactionBuffer.getMaxReadPosition()
, which filters uncommitted transactional writes. For non-transactional publishes this is still equivalent to LAC, but in-flight txn messages won't be visible. • The Java
Reader.getLastMessageId()
/
getLastMessageIdAsync()
are deprecated in favor of
getLastMessageIdsAsync()
(returns
List<TopicMessageId>
). Same broker semantics, just a more partition-friendly API shape. Go client hasn't followed that rename. TL;DR: publish, then query — you're safe. The sentinel pattern is only needed when you can't trust that the broker you're querying has seen the write (e.g., cross-cluster geo-replication, where the ack came from the source cluster but you're querying the replica).
-------------------------------- The answer above was created with Claude Desktop and a prompt "Answer this question about Apache Pulsar using DeepWiki MCP searching apache/pulsar and apache/pulsar-client-go".
j
Thanks, Lari. As likely expected, I had asked LLMs, and while their answers pointed in the same direction, none of them could direct me to an authoritative source. It feels like something that should be found in the docs (it's always possible that I failed to find it).
On the above, I'm certain that PRs are welcome and I'm happy to amend the docs in case I didn't fail to find this info 🙂
l
It feels like something that should be found in the docs (it's always possible that I failed to find it)
I agree. Would you like to create a PR to https://github.com/apache/pulsar-client-go or https://github.com/apache/pulsar-site to improve the docs?
j
Absolutely, I'm on it 👍
thankyou 1
Hey @Lari Hotari, I'll eventually get to the docs this week (yesterday I opened a PR fixing a bug: https://github.com/apache/pulsar-client-go/pull/1482). But I have another question, related to the previous one. Do I enjoy the same guarantees - i.e., see the ID of the last message in the topic - if I create the reader before acquiring the exclusive producer using the same client?
l
Do I enjoy the same guarantees - i.e., see the ID of the last message in the topic - if I create the reader before acquiring the exclusive producer using the same client?
Yes, since the GetLastMessageID will trigger a RPC call over the Pulsar protocol.
j
Perfect, thank you!