https://bentoml.com logo
Join Slack
Powered by
Hello, I'm having trouble setting up observability...
# ask-for-help
s
Hello, I'm having trouble setting up observability for my Bento service to view metrics and logs related to my service. I've read and re-read the docs but it's unclear to me how to really proceed in order to fetch and view them. Please help.
This is how my service is structured so far: 
import bentoml
import numpy as np
@bentoml.service(workers=2)
class InputClassifier:
bento_model = bentoml.models.get(
"random_forest_large_binary_classifier:2mzuac2e22sq2aam"
)
def __init__(self):
self.model = bentoml.mlflow.load_model(self.bento_model)
@bentoml.api
def predict(self, input_data: np.ndarray) -> np.ndarray:
with bentoml.monitor("incoming predictions") as mon:
mon.log(input_data, name="Incoming input data", role="input", data_type="numerical")
return self.model.predict(input_data)
j
Can you share what are you trying to achieve here and which part is not working as expected
s
So I have my predict function which just returns my model's prediction that I'm serving with bento as a service (all done locally). I want to: • Be able to log the incoming input data in the requests I'm getting. • The response-time for the incoming requests. • Current number of requests being handled (since I might have more than one worker handling incoming requests). Just overall service-related metrics that I can access through both log files and possibly prometheus or graffana. I know bento has this option but I wasn't able to make it work or even understand how I can collect and see my logs from the observability section in the docs.
j
• Be able to log the incoming input data in the requests I'm getting.
Where is intended destination of your logs? with your current code you should see a local log file created.
• The response-time for the incoming requests.
• Current number of requests being handled (since I might have more than one worker handling incoming requests).
This is available in the 
/metrics
endpoint
👍🏻 1
s
Okay, I'm able to see the logs when I run the command 
curl -o metrics.txt ip_address:port/metrics
for my service. (I'm working on a bare bone server with no GUI at the moment) At least now I can see the metrics. I was following this part of the documentation and not understanding where my 
.log
file was or how could I generate it: Deploy the service and collect monitoring data in https://docs.bentoml.com/en/v1.1.11/guides/monitoring.html
Could I follow this article in order to monitor these metrics through Prometheus even though it says deprecated? https://www.bentoml.com/blog/monitoring-metrics-in-bentoml-with-prometheus-and-grafana
j
did you see a 
monitoring
or 
data
folder being created locally?
s
No neither. I create my service locally by first running bentoml build --containerize and then running a container from my created image. No new folders appeared.
j
ah, because the folder is created in the container, those files appears INSIDE your container. if you want it to appear locally, you will need to mount a volume from your local path to the container
s
Ahh that's right, I'll try it out Thank you !
I found the monitoring folder inside the docker container. I've tried to run: 
docker run --rm -p 3000:3000 -v ./files/bento-monitoring:/src/monitoring signal_classifier:fy5p3bsla6qysm
to set up a volume for any files that get created inside the monitoring folder to be copied to my host machine in 
files/bento-monitoring
directory. However nothing is happening. Should I mount to just 
/src
instead ? or go a deeper when setting up the volume with 
/src/monitoring/<name>/data
inside the bento container ?
2 Views
Open in Slack
PreviousNext
Powered by