This message was deleted.
# general
s
This message was deleted.
l
The Pulsar Grafana dashboards are useful for monitoring. In the next version of Apache Pulsar Helm chart, Pulsar Grafana dashboards from https://github.com/streamnative/apache-pulsar-grafana-dashboard will get preinstalled. When there's a backlog for a subscription, the messages will be preserved. The disk space won't become available until Bookkeeper has done data compaction (or gc), the description is here: https://bookkeeper.apache.org/docs/getting-started/concepts#data-compaction Here are some better defaults for Apache Pulsar Helm chart Bookkeeper that will help disk from filling up (assumes that there are no backlog that prevents ledger deletion):
Copy code
bookkeeper:
  configData:
    #
    # Bookkeeper configuration reference: <https://bookkeeper.apache.org/docs/reference/config>
    #
    # configure the data compaction (bookie entry log compaction and gc) settings
    # <https://bookkeeper.apache.org/docs/reference/config#garbage-collection-settings>
    # <https://bookkeeper.apache.org/docs/reference/config#entry-log-compaction-settings>
    minorCompactionThreshold: "0.2"     # default 0.2 (use default)
    minorCompactionInterval: "360"      # default 3600 seconds (6 minutes vs default 1 hour)
    majorCompactionThreshold: "0.8"     # default 0.5
    majorCompactionInterval: "10800"    # default 86400 seconds (3 hours vs default 1 day)
    gcWaitTime: "300000"                # default 900000 milli-seconds (5 minutes vs default 15 minutes)
    isForceGCAllowWhenNoSpace: "true"   # default false
    # disk utilization configuration
    # <https://bookkeeper.apache.org/docs/reference/config#disk-utilization>
    # Make sure that diskUsageLwmThreshold <= diskUsageWarnThreshold <= diskUsageThreshold
    diskUsageLwmThreshold: "0.85"   # default 0.90 
    diskUsageWarnThreshold: "0.9"   # default 0.95
    diskUsageThreshold: "0.95"      # default 0.95 (use default)
    diskCheckInterval: "1800"       # default 10000
This config change is part of https://github.com/apache/pulsar-helm-chart/pull/454
👍 1
You might also have to do some other tuning on Pulsar side. For load tests there are many details that need tuning.