Hello. I keep getting errors like

http: panic serving 10.1.5.27:50774: Get "<http://localhost:3000/metrics>": read tcp 127.0.0.1:58718->127.0.0.1:3000: read: connection reset by peer

all throughout my bentoml cloud logs. I am often using the async pattern as described here (https://github.com/bentoml/BentoML/issues/4148) so I suspect it may have something to do with that but Im not sure. I need a way to resolve that error or, at the very least, turn off the metrics for now if anyone can point me how to do that? Thanks!

Hi @Ed Dean, the reason you see the panic log for the

/metrics

endpoint is because you are probably executing blocking logic, like model inference, on the async event loop. Blocking the event loop is not recommended because critical IO operations such as metrics and health checks won’t get executed in time, potentially causing the deployment to fail. For blocking tasks to be run on an async endpoint, the recommended pattern is to execute blocking logic on separate threads using utilities like

asyncio.run_in_executor

.

gotcha. ill try that out. Thanks @Sean