This message was deleted.

It's not a type hint, it's the class that BentoML uses to construct the input from the HTTP request.

Oh I see - I misread that. Thanks for clearing it up!

I'm not super sure what you mean by that; we need to know what we're the request body to and how the conversion should be done, which is what the IO descriptor provides.

OK I must not be understanding what BentoML does under the hood - I meant for example using something like Fast API, it can handle any request body and response since it’s a generic server.

You could do that yourself with a custom IO descriptor (or just a bytes input), but BentoML is for ML applications, and most users prefer to have their input just be an ndarray that they can immediately use instead of doing the conversion themselves.

OK so it’s mostly for convenience that you provide these classes

A lot of the goal when designing BentoML was to make the framework as easy to use for data scientists as possible, so the idea is that they don't have to understand that there's an underlying request at all. As a bonus, it also allows us to support both HTTP and gRPC servers with a single API endpoint definition.

Right - I think I get it now. Thanks for explaining this!

👍 no problem!