Apache Pinot

Hi Team :wave: , We have a Pinot cluster with jmx exporter configured. However, on the controllers, we can only see `ValidationMetrics`  getting exposed along with some jvm metrics and not the `ControllerMetrics` . We have been using the jmx config file from the apache/pinot repository - <https://github.com/apache/pinot/blob/4f17ede26c3a5c92dea5106654e685a2a411287d/docker/images/pinot/etc/jmx_prometheus_javaagent/configs/pinot.yml|pinot.yml> and want to keep a track of metrics like `SEGMENTS_IN_ERROR_STATE`  but they do not get exposed by the jmx exporter at the defined port. Can I please get some help in figuring out how all the metrics can be exported.
```- pattern: "\"org.apache.pinot.common.metrics\"&lt;type=\"ControllerMetrics\", name=\"pinot.controller.segmentsInErrorState.(\\w+)_(\\w+)\"&gt;&lt;&gt;(\\w+)"
  name: "pinot_controller_segmentsInErrorState_$3"
  labels:
    table: "$1"
    tableType: "$2"```

This yaml file should work. <@UKH768C4D>?

:wave: Any inputs on this one?
I can see the `SegmentStatusChecker` periodic task running, which manages segment status metrics, but ControllerMetrics like `SEGMENTS_IN_ERROR_STATE` do not get exposed
```2022/08/04 02:05:09.414 INFO [PeriodicTaskScheduler] [pool-9-thread-1] Starting SegmentStatusChecker with running frequency of 300 seconds.
2022/08/04 02:05:09.414 INFO [BasePeriodicTask] [pool-9-thread-1] [TaskRequestId: auto] Start running task: SegmentStatusChecker
2022/08/04 02:05:09.415 INFO [ControllerPeriodicTask] [pool-9-thread-1] Processing 1 tables in task: SegmentStatusChecker```

This is fixed. There was an error in the jmx config file that is being used and we identified it. :thankyou:

Could you elaborate the issue and the fix? Was it something on the open source side or specific to your deployment?

It was on our end of deployment. Our config file was missing some periods (`.`) in the rules and hence the metrics weren't exposed.

<@UDQU92KBK> Do you know if Pinot has any metrics to track the segments in `Bad` state. In the UI, we see some segments in `Bad`  state and wanted to find out if they can be tracked and alerted.

I believe there is percent replicas up, iirc 