https://pinot.apache.org/ logo
j

Josh Highley

03/12/2021, 6:38 PM
when ingesting large amounts of streaming data, is there any mechanism for capturing/notifying about data errors other than looking through the server log files? We're ingesting millions of records as fast as Pinot can ingest them, so looking through logs isn't really practical.
d

Daniel Lavoie

03/12/2021, 6:40 PM
Please share your use case to this issue: https://github.com/apache/incubator-pinot/issues/6524
j

Josh Highley

03/12/2021, 6:43 PM
that seems to be at a table level -- connectivity issues or configuration issues. Is row-level monitoring applicable to that issue?
d

Daniel Lavoie

03/12/2021, 6:44 PM
It’s worth sharing as this bring another perspective to the general “ingestion” observability topic.
If one row breaks ingestion, it’s equaly important to bubble up the details than a table misconfiguration.
I think both issues fits in the “table ingestion status” story
k

Kishore G

03/12/2021, 7:23 PM
there is a metric that you can monitor for errors during ingestion
j

Josh Highley

03/12/2021, 9:27 PM
I see an error count in ServerMeter but it's just a counter. Per instructions here: https://docs.pinot.apache.org/operators/operating-pinot/monitoring#customizing-metrics I exposed the metrics but the counter is still 0 despite errors in the logs