Stanislaw Makarawec
06/27/2025, 3:31 PMsubscriptionExpirationTimeMinutes in broker.conf but unactive subscriptions wasn't deleted. Pulsar 4.0.4. What should I do, for auto cleanup subscriptions without active consumers?RunLLM
06/27/2025, 3:31 PMsubscriptionExpirationTimeMinutes in broker.conf enables automatic deletion of inactive subscriptions, but this only works for topics that are currently loaded in the broker's memory. If a topic is not loaded (no recent activity), the broker will not check or delete its inactive subscriptions until the topic is reloaded (e.g., by publishing or consuming a message) (GitHub Issue #19968). This is a known limitation and there is no broker setting to force periodic loading of all topics for cleanup.
Would you like more detail on workarounds or internal logic?
Sources:
- GitHub Issue #19968 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.Stanislaw Makarawec
06/27/2025, 3:34 PMRunLLM
06/27/2025, 3:34 PMsubscriptionExpirationTimeMinutes, it will be deleted (GitHub Issue #19968).
Would you like more detail on the exact logic or configuration?
Sources:
- GitHub Issue #19968 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.Stanislaw Makarawec
06/27/2025, 4:16 PMsubscriptionExpirationTimeMinutes=5
subscriptionExpiryCheckIntervalInMinutes=1
If there was no action within the subscription, it will not be deleted as well?RunLLM
06/27/2025, 4:16 PMStanislaw Makarawec
06/27/2025, 4:22 PMsubscriptionExpirationTimeMinutes=5
subscriptionExpiryCheckIntervalInMinutes=1
stats
k8s-pulsar-dev-broker-0:/pulsar$ ./bin/pulsar-admin topics stats <persistent://server_smakaravets/toll_gates_cache/global>
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesInCounter" : 150,
"msgInCounter" : 3,
"systemTopicBytesInCounter" : 0,
"bytesOutCounter" : 13200,
"msgOutCounter" : 264,
"bytesOutInternalCounter" : 0,
"averageMsgSize" : 0.0,
"msgChunkPublished" : false,
"storageSize" : 150,
"backlogSize" : 150,
"backlogQuotaLimitSize" : -1,
"backlogQuotaLimitTime" : -1,
"oldestBacklogMessageAgeSeconds" : -1,
"publishRateLimitedTimes" : 0,
"earliestMsgPublishTimeInBacklogs" : 0,
"offloadedStorageSize" : 0,
"lastOffloadLedgerId" : 0,
"lastOffloadSuccessTimeStamp" : 0,
"lastOffloadFailureTimeStamp" : 0,
"ongoingTxnCount" : 0,
"abortedTxnCount" : 0,
"committedTxnCount" : 0,
"publishers" : [ ],
"waitingPublishers" : 0,
"subscriptions" : {
"eaa16c41-5248-44f9-bb90-8eb0d162de00" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 13150,
"msgOutCounter" : 263,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 3,
"backlogSize" : 150,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 3,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1751040069465,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"replicated" : true,
"durable" : true
},
"test" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 50,
"msgOutCounter" : 1,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 1,
"backlogSize" : 50,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 1,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1751039943370,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 1751039979675,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : false,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"replicated" : false,
"durable" : true
},
"82af7147-ffb6-416f-857f-6dd61eddc49e" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 3,
"backlogSize" : 150,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 3,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "None",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"replicated" : true,
"durable" : true
},
"e5ce8be7-d0bb-4e92-8bdb-eb2d94e8e3c2" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 3,
"backlogSize" : 150,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 3,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "None",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"replicated" : true,
"durable" : true
},
"f06584b0-ded7-4fe9-9d5c-b5acbd46bec7" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 3,
"backlogSize" : 150,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 3,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "None",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"replicated" : true,
"durable" : true
}
"replication" : { },
"deduplicationStatus" : "Disabled",
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"compaction" : {
"lastCompactionRemovedEventCount" : 0,
"lastCompactionSucceedTimestamp" : 0,
"lastCompactionFailedTimestamp" : 0,
"lastCompactionDurationTimeInMills" : 0
},
"ownerBroker" : "k8s-pulsar-dev-broker-2.k8s-pulsar-dev-broker.pulsar-dev.svc.cluster.local:8080"
}
Subscriptions still existRunLLM
06/27/2025, 4:22 PMsubscriptionExpirationTimeMinutes and subscriptionExpiryCheckIntervalInMinutes are set (GitHub Issue #19968). In your stats, the topic is idle, so subscriptions remain.
Would you like more detail on how to force cleanup or monitor topic loading?
Sources:
- GitHub Issue #19968 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.Stanislaw Makarawec
06/27/2025, 4:25 PMRunLLM
06/27/2025, 4:25 PMownerBroker field in topic stats indicates which broker is responsible for the topic, but it does not guarantee the topic is currently loaded in broker memory. A topic can have an owner broker assigned but still be unloaded (not active in memory) (Pulsar topic stats docs, GitHub PR #17136).
Would you like more detail on how to check if a topic is actually loaded?
Sources:
- Pulsar topic stats docs
- GitHub PR #17136 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.Stanislaw Makarawec
06/27/2025, 4:25 PMRunLLM
06/27/2025, 4:25 PMStanislaw Makarawec
06/27/2025, 4:37 PMRunLLM
06/27/2025, 4:37 PMsubscriptionExpirationTimeMinutes are eligible for automatic deletion (GitHub Issue #19968).
Would you like more detail on how the expiration logic works for subscriptions without consumers?
Sources:
- GitHub Issue #19968 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.Stanislaw Makarawec
06/27/2025, 4:39 PM"subscriptions" : {
"eaa16c41-5248-44f9-bb90-8eb0d162de00" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 81800,
"msgOutCounter" : 1636,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 4,
"backlogSize" : 200,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 4,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1751042211692,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"durable" : true,
"replicated" : true
},
"test" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 50,
"msgOutCounter" : 1,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 0,
"backlogSize" : 0,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 0,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"activeConsumerName" : "3ouQl",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1751041915097,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 1751041769802,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"consumerName" : "3ouQl",
"availablePermits" : 1000,
"unackedMessages" : 0,
"avgMessagesPerEntry" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"address" : "/10.233.118.166:52092",
"connectedSince" : "2025-06-27T16:31:55.096010768Z",
"clientVersion" : "Pulsar-Java-v4.0.4",
"lastAckedTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1751041915097,
"metadata" : { },
"lastAckedTime" : "1970-01-01T00:00:00Z",
"lastConsumedTime" : "1970-01-01T00:00:00Z"
} ],
"isDurable" : true,
"isReplicated" : false,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"durable" : true,
"replicated" : false
},
"82af7147-ffb6-416f-857f-6dd61eddc49e" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"msgBacklog" : 4,
"backlogSize" : 200,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 4,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"msgInReplay" : 0,
"unackedMessages" : 0,
"type" : "None",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"lastMarkDeleteAdvancedTimestamp" : 0,
"consumers" : [ ],
"isDurable" : true,
"isReplicated" : true,
"allowOutOfOrderDelivery" : false,
"consumersAfterMarkDeletePosition" : { },
"drainingHashesCount" : 0,
"drainingHashesClearedTotal" : 0,
"drainingHashesUnackedMessages" : 0,
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0,
"delayedMessageIndexSizeInBytes" : 0,
"subscriptionProperties" : { },
"filterProcessedMsgCount" : 0,
"filterAcceptedMsgCount" : 0,
"filterRejectedMsgCount" : 0,
"filterRescheduledMsgCount" : 0,
"dispatchThrottledMsgEventsBySubscriptionLimit" : 0,
"dispatchThrottledBytesEventsBySubscriptionLimit" : 0,
"dispatchThrottledMsgEventsByTopicLimit" : 0,
"dispatchThrottledBytesEventsByTopicLimit" : 0,
"dispatchThrottledMsgEventsByBrokerLimit" : 0,
"dispatchThrottledBytesEventsByBrokerLimit" : 0,
"durable" : true,
"replicated" : true
},RunLLM
06/27/2025, 4:39 PM"eaa16c41-5248-44f9-bb90-8eb0d162de00" and "82af7147-ffb6-416f-857f-6dd61eddc49e" have no active consumers ("consumers": <https://github.com/apache/pulsar/issues/19968|]). These are eligible for auto-deletion if they remain inactive for longer than subscriptionExpirationTimeMinutes and the topic is loaded in broker memory ([GitHub Issue #19968>). If they are not being deleted, it is likely because the topic is not loaded or the expiration check has not run since they became inactive.
Would you like more detail on how to force the expiration check or confirm topic loading?
Sources:
- GitHub Issue #19968 π 0 button π 0 button π‘ Hint: Mention @runllm in the thread for followups.