Hi. Is there any way from the rest API to retreive informations to monitor like nb_messages read by consumer / nb messages indexed . The goal here in my question is to monitor the ingestion and ensure we are not missing messages. I’ve found messages like that on the pinot-all.log but I want them from API if possible. Any recomanded way ?

there’s no API as of now, but you could just monitor the metrics emitted

REALTIME_ROWS_CONSUMED("rows", true),
  INVALID_REALTIME_ROWS_DROPPED("rows", false),
  REALTIME_CONSUMPTION_EXCEPTIONS("exceptions", true),
  REALTIME_OFFSET_COMMITS("commits", true),
  REALTIME_OFFSET_COMMIT_EXCEPTIONS("exceptions", false),
  REALTIME_PARTITION_MISMATCH("mismatch", false),
  ROWS_WITH_ERRORS("rows", false),

If you want to see the same metrics that you’re seeing in the logs (which are just for the scope of the consuming segment, not overall) that should be easy to add to an existing /consumingSegmentsInfo API. Do you mind filing a GH issue?

Ok thanks for you feedback. I will look arround for metrics JMX metrics 😉 And look how it works. Many thanks