This message was deleted BentoML #ask-for-help

Join Slack

This message was deleted.

# ask-for-help

Slackbot

01/31/2023, 8:09 AM

This message was deleted.

👀 1

Sean

01/31/2023, 9:22 AM

Thanks for the insightful post, @Eric Riddoch. It seems like you are integrating with newrelic through the Python Agent. Haven’t read a lot into the implementation, it seems like the instrumentation is automatically injected into the ASGI application if the application was started with

newrelic-admin run-program uvicorn

. Treating BentoML as a simple ASGI application will not work out-of-the-box. BentoML’s runner architecture spawns multiple processes to optimize system resource utilization. Instead of a single uvicorn process, BentoML consists of multiple ASGI processes. I will need to look into New Relic further to investigate a better integration. Since BentoML is fully compliant to OpenTelemetry and Prometheus standards, I’m hopeful that New Relic has proper supports for those standards.

Sean

01/31/2023, 9:30 AM

I’m not seeing any evidence of errors, even though most of my requests were to the
/newrelic-error
endpoint

I think this is due to New Relic not recognizing

bentoml.exceptions.InternalServerError

No trace ID in the response headers; with FastAPI the response headers include the trace ID. FastAPI also knows how to handle when the trace ID is present in the request headers

As discussed in an earlier thread, a better approach I think is to pass through callers trace context so the caller knows the Trace ID implicitly when errors happen. I think the gist of all these issues is the Python Agent. Again, I believe a better integration with New Relic is just around the corner.

Eric Riddoch

01/31/2023, 9:33 AM

If we could take advantage of Bento's already existing setup with prometheus and OpenTelemetry, that would be awesome!

Sean

01/31/2023, 9:53 AM

New Relic supports OpenTelemetry Protocol for exporting trace data. In BentoML, you can customize the OTLP exporter in the Tracing configuration with New Relic’s endpoint.

Sean

01/31/2023, 9:54 AM

New Relic supports exporting Prometheus metrics through their Prometheus OpenMetrics integration for Docker.

Sean

01/31/2023, 10:00 AM

Happy to work closer on a POC if you’d like to explore further. 🙂

Eric Riddoch

01/31/2023, 10:02 AM

I'd love that! I'm liking the sound of the OpenMetrics integration for docker. If that's a subprocess that we can run in the container that hits the

/metrics

endpoint on a schedule and publishes those to NewRelic, that would be perfect.

Eric Riddoch

01/31/2023, 10:02 AM

Setting up the OpenTelemetry one is exciting, too. I'm not an OTel expert, so I'm not sure how tha tintegration would go

Eric Riddoch

01/31/2023, 10:03 AM

We could sign up for a trial new relic account and put a trivial, dockerized sample project in an open source repo--maybe under the bento organization

Eric Riddoch

02/06/2023, 10:39 PM

Small update on this: https://github.com/bentoml/BentoML/issues/3463#issuecomment-1419879591

4 Views

Open in Slack

Previous Next