This message was deleted.

This is probably more of a

uvicorn

feature. BentoML uses starlette with

uvicorn

for both runner and api server. and uvicorn has graceful shutdown by default. https://www.uvicorn.org/server-behavior/#graceful-process-shutdown however, if you need additional control of the lifecycle of the server, you can do it with hooks. https://docs.bentoml.com/en/latest/concepts/service.html#lifecycle-hooks