This message was deleted.
# announcementss
Slackbot
09/30/2020, 3:56 AMThis message was deleted.
h
Hirako Sa
09/30/2020, 3:57 AMI see that both predictions and feedbacks are printed in log files on the filesystem. Is there a consideration to rather send those, via configuration, to a centralised store?
Hirako Sa
09/30/2020, 3:58 AMThe same way some network metrics are gathered by prometheus, there is value in gathering predictions (outcome of the models) and feedbacks (income to the models) to an aggregator for analysis. Right now we would have to parse the logs on X number of servers.
c
Chaoyu
09/30/2020, 4:42 AMHi Hirako, log collection can be challenging when working with high volume prediction services, that’s why we left it out-of-scope for BentoML, and makes it easy to work with log collection tools like logstash and fluentd. Although those tools are not ideal for data scientist to access and use those data for analytics or model development.
In my opinion, this is something very hard to offer as an open source component that’s applicable for everyone, but could be part of our commercial product (a model deployment lifecycle management platform on top of BentoML)
h
Hirako Sa
09/30/2020, 4:50 AMOK. currently using fluentbit (similar to fluentd). Prediction and feedback is not really logging though.
Hirako Sa
09/30/2020, 4:51 AMif I follow your logic (dedicating the centralized info to things like fluentd), then the predictions should be printed to a different appender. the same way feedback entries are appended to a feedback specific appender.
Hirako Sa
09/30/2020, 4:52 AMwith that, we can have fluentd or other aggregator taking care of it. as it stands it would aggregate all prediction outcomes along with the rest of the server logging. (irrelevant for analysis of predictions).