This message was deleted.

Hi @Ashish Singh - you don’t need to add the @svc.api decorator to your FastAPI’s router - you can actually call

runner.run

or

runner.async_run

from your fastapi endpoints. The only thing you need to do to ensure the Runner architecture is applicable: 1. make sure to pass all runner references to the Service instance, e.g.

svc = bentoml.Service("MyService", runners = […])

2. make sure to use

bentoml serve

to launch the service FastAPI in this case only runs in the API server processes, along side with other

svc.api

logic if you have defined any.

So there is no linkage between an InferenceAPI and runner from a decorator standpoint If I defined my svc as such

svc = bentoml.Service("MyService", runners = […])

and then mounted the fastapi svc to this svc Okay, I was worried if I don't define the decorator, the svc inference won't work, since i use runner inside fastapi route 🤦 Thank you so much for clearing that out for me 🙏

No problem Ashish! Happy to help. We probably should have included an example for that use case.

Yeah it only says we can mount the Fastapi app, but how does that work with bento internally is something I got lost in 😅