Hello everyone, I have a question regarding error handling. Is there a way to add exception handlers similar to those in FastAPI? I've tried adding exception handlers to the mounted FastAPI app, but they don't seem to catch exceptions raised from the BentoML service method (decorated with

@bentoml.api

). Thanks!

Sounds like you’ve added an exception handling middleware to the FastAPI app mounted to BentoML service - what you can do is mount the MiddleWare to BentoML service instead: https://docs.bentoml.com/en/latest/build-with-bentoml/asgi.html#add-custom-asgi-middleware

What am I doing wrong here?

from __future__ import annotations

from typing import Any

import bentoml
from bentoml.exceptions import BadInput
from pydantic import BaseModel
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import JSONResponse, Response


class ErrorHandlingMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next) -> Response:
        try:
            return await call_next(request)
        except BadInput as e:
            return JSONResponse(
                status_code=400, content={"error": "some custom error"}
            )
        except Exception as e:
            return JSONResponse(
                status_code=500, content={"error": "some custom error"}
            )


class InputData(BaseModel):
    """Input data for the service."""

    data: int


@bentoml.service
class SimpleService:
    @bentoml.api
    async def predict(self, input_data: InputData) -> dict[str, Any]:
        """Predict the output for the given input data."""
        if input_data.data > 10:
            raise BadInput("Input data is too large")
        return {"data": input_data.data * 2}


SimpleService.add_asgi_middleware(ErrorHandlingMiddleware)

I expected to get:

{
  "error": "some custom error"
}

But instead, i get:

[
  {
    "error": "Input data is too large"
  }
]

Alternatively, a solution for handling errors directly in the API route would also be great. Here's an example from FastAPI documentation:

@app.exception_handler(UnicornException)
async def unicorn_exception_handler(request: Request, exc: UnicornException):
    return JSONResponse(
        status_code=418,
        content={"message": f"Oops! {exc.name} did something. There goes a rainbow..."},
    )