This message was deleted.
# generals
Slackbot
04/14/2023, 9:16 AMThis message was deleted.
y
Yu Wei Sung
04/14/2023, 1:36 PMYu Wei Sung
04/14/2023, 1:36 PMcheck the data compaction part.
Yu Wei Sung
04/14/2023, 1:40 PMThe ledger disk only reclaimed when compaction kicks in.
m
Mohd Hidayathulla
04/14/2023, 1:44 PMThank you for the reply, I will go through this document and check. Meanwhile the disk consumption is very high once it reaches 90% we are getting "no non faulty bookkeepr is available" and the flow stops.
y
Yu Wei Sung
04/14/2023, 1:50 PMthat’s retention. if you have topics without any consumers/subscriptions, the default behavior is retent forever. to troubleshoot, check backlogs of topic stats.
d
David K
04/14/2023, 2:37 PMAnother thing to check is for any open subscriptions on topics inside the namespace. Any messages on a subscription will not be deleted, and thus supersedes any retention policies you may have. The assumption is that an subscription indicates that a consumer intends to return and resume processing.
David K
04/14/2023, 2:39 PMIf you find that is this is the cause of the your data retention issue, you can configure Pulsar to expire subscriptions that have no consumers connected using the
subscriptionExpirationTimeMinutesbroker.confbin/pulsar-admin namespaces set-subscription-expiration-time --time 60m
Mohd Hidayathulla
04/20/2023, 6:35 AMI tried cleaning up the subscription and also the backlog but retention is still not working and there is a constant increase in the disk usage. I have implemented TTL in the lower environment ost that there is not spike in disk space. So my question is can I have retention policy and TTL both or should I go with only TTL.
d
David K
04/20/2023, 2:25 PMYou can have both TTL and data retention policies in place at the same time. TTL applies to unacknowledged messages, and data retention applies to acknowledged messages.
2 Views