@RunLLM will metrics related to partitioned topics always be emitted at the level of each partition? Is it possible to instead get metrics at the topic level instead of partitioned topic?

By default, Pulsar emits metrics for partitioned topics at the partition level, which can result in high cardinality (many metrics) if you have many partitions. However, you can configure Pulsar to emit metrics at the topic level (aggregated across all partitions) instead of per-partition. This is controlled by the

exposeTopicLevelMetricsInPrometheus

setting in the broker configuration; setting it to

false

will aggregate metrics at the namespace level, while

true

enables topic-level metrics, including partitioned topics. You can also split topic and partition labels for Prometheus metrics if needed. For more details, see the configuration options in ServiceConfiguration.java and the discussion in this GitHub issue. Would you like more detail on how to configure or query these metrics? Sources: - ServiceConfiguration.java - Make Topic Level Prometheus Metrics Optional 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@RunLLM that does not seem right. If I set

exposeTopicLevelMetricsInPrometheus

to

false

, it will emit metrics at namespace level which is not what I want

You are correct. Setting

exposeTopicLevelMetricsInPrometheus

to

false

emits metrics at the namespace level, not at the topic level. If you want metrics aggregated at the topic level (i.e., for the logical topic, not each partition), Pulsar does not natively aggregate partitioned topic metrics into a single topic-level metric; it emits metrics for each partition separately when

exposeTopicLevelMetricsInPrometheus

is

true

 — there is no built-in option to aggregate all partitions of a topic into a single topic-level metric in the Prometheus output. For reference, see the behavior described in Pulsar metrics documentation and the configuration in ServiceConfiguration.java. Would you like more detail on possible workarounds or external aggregation? Sources: - Pulsar metrics documentation - ServiceConfiguration.java 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.