Mounting a FastAPI asgi app on

path="/"

destroys the open API documentation and gives "Not found. E.g. This simple API:

import bentoml
from fastapi import Depends, FastAPI

app = FastAPI()


@bentoml.service(
    name="The_cake_is_a_lie",
)
@bentoml.mount_asgi_app(app) # Only works with path="/"
class Service:
    @bentoml.api(output_spec=dict)
    def classify(self) -> dict:
        return {"Cake": None}


@app.get("/metadata", tags=["Metadata"])
def metadata():
    return {"metadata": "Yay"}


if __name__ == "__main__":
    Service.serve_http(port=3000)

I have services that calls asgi endpoints on root, like "/some_statistics", etc, but it only works if they get their own path. Is this intended or can I circumvent it somehow?

typically we recommend mounting to a specific prefix `from fastapi import FastAPI, Depends import bentoml app = FastAPI() @bentoml.service @bentoml.mount_asgi_app(app, path="/v1") class MyService: name = "MyService" @app.get('/hello') def hello(self): # Inside service class, use

self

to access the service return f"Hello {self.name}"`

But it seems more of a requirement than a recommendation? I read some year old issues/tickets/prs on github about a metadata endpoint. Was that never implemented?

Not a requirement, really just depending on what you need - see the question above about disablimg the openapi page :)

Stuff like this:

@app.get("/metadata", tags=["Metadata"])
def metadata(service: XrayService = dependable_service):
    mlflow = service.bentoml_model.custom_objects["mlflow"]
    # We convert nans to -1 to avoid serialization issues
    metrics = mlflow["metrics"]
    metrics = {k: -1 if np.isnan(v) else v for k, v in metrics.items()}
    mlflow["metrics"] = metrics
    return {
        "name": service.bentoml_model.tag.name,
        "version": service.bentoml_model.tag.version,
        "config": service.bentoml_model.custom_objects["config"],
        "output_schema": service.bentoml_model.custom_objects["output_schema"],
        "mlflow": mlflow,
    }

Anything mlflow related, training/evaluation metrics, version etc. Making sure that the model i containerized is actually the one being served 🙂

Got it

But @api becomes a post endpoint, right?

Correct

Alright, I know my options then. I just thought "metadata" was universal enough to be hosted alongside metrics, livez, healthz etc

Np! We thought about that idea but unfortunately there isn’t a universal “metadata” that makes sense across different type of ml/ai services and we think it’s best to give users the flexibility to decide what they want to expose