This message was deleted.

Your expectation is correct, in the second example there should be no message returned for k1

userProducer.newMessage().keyBytes(("key"+i).getBytes(StandardCharsets.UTF_8))
         .value(msg1.getBytes(StandardCharsets.UTF_8)).sendAsync();

is the example produce and compaction is triggered as below using pulsar admin

pulsarAdmin.topics().triggerCompaction(topic);

Strange, the code looks fine to me

@sindhushree You can check the topic's internal stats to see how many entries are in the compacted ledger and how many entries are in the original topic.

@Penghui Compacted ledger has 3 entries and when i read am getting 3 entries .but on the subscription am getting this ack holes. "test" : { "markDeletePosition" : "66:-1", "readPosition" : "169:12", "waitingReadOp" : false, "pendingReadOps" : 0, "messagesConsumedCounter" : 80, "cursorLedger" : 174, "cursorLedgerLastEntry" : 0, "individuallyDeletedMessages" : "[(678..6711],(731..734],(762..7611],(857..8510],(1491..14910],(14913..14916],(1591..1594],(1652..1655],(1692..1695],(1698..16911]]", "lastLedgerSwitchTimestamp" : "2024-01-29T063126.116Z", "state" : "Open", "numberOfEntriesSinceFirstNotAckedMessage" : 91, "totalNonContiguousDeletedMessagesRange" : 10, "subscriptionHavePendingRead" : false, "subscriptionHavePendingReplayRead" : false, "properties" : { } }

Which subscription type are you using? only the failover or exclusive subscription types are supported on reading compacted data

userConsumer = client.newConsumer()
      .topic(topic)
      .subscriptionName("test")
      .subscriptionType(SubscriptionType.Exclusive)
      //.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
      .subscriptionMode(SubscriptionMode.Durable)
      .readCompacted(true)
      .subscribe();

its exclusive and i had subscription position set to earliest

Interesting. Do you have transaction enabled or replicated subscription enabled? Or could you please share the complete the topic stats and internal-stats?

we dont have transaction enabled nor replicated subscription .

Ok, can you reproduce the issue? Please help create a GitHub issue? From your description, the behavior doesn't look expected. We might need to investigate more on the Github issue.

I can create a github issue with sample produce and consume code along with stats and stats internal

Yes, Please help create an issue and I will investigate it.

@sindhushree For readcompacted=true，I think you should use acknowledgeCumulative to avoid ack holes.

in doc its not mentioned ..so is the ack indivisual is the suspect ?

@Cong Zhao I think we need to improve the code or the documentation. If we’d better disable the individual ack for read compacted, it’s better to have a discussion under the mailing list.

@Penghui @Cong Zhao cumulative ack is working ..we dont see ack holes

Thanks for confirming.