when ingesting large amounts of streaming data, is there any mechanism for capturing/notifying about data errors other than looking through the server log files? We're ingesting millions of records as fast as Pinot can ingest them, so looking through logs isn't really practical.

Please share your use case to this issue: https://github.com/apache/incubator-pinot/issues/6524

that seems to be at a table level -- connectivity issues or configuration issues. Is row-level monitoring applicable to that issue?

It’s worth sharing as this bring another perspective to the general “ingestion” observability topic.

there is a metric that you can monitor for errors during ingestion

I see an error count in ServerMeter but it's just a counter. Per instructions here: https://docs.pinot.apache.org/operators/operating-pinot/monitoring#customizing-metrics I exposed the metrics but the counter is still 0 despite errors in the logs