https://bentoml.com logo
Join Slack
Powered by
Hi! My API EP looks like this: ```@bentoml.api asy...
# ask-for-help
a
Hi! My API EP looks like this:
Copy code
@bentoml.api
async def generate(self, input_request: GenerateImageRequestBase) -> GenerateImageResponseBase:

    app_logger.info(f"request type {TYPE} for {input_request.studio_name} with {input_request}")

    # prepare model params
    model_params = map_params_to_pipeline(input_request)
where GenerateImageRequestBase is pydantic class swagger seems to generate the following request example:
Copy code
{
  "input_request": {
    "modelId": "pixel-art-xl-lora",
    "type": "character",
    "prompt": "a 3d rendering",
    "numOfImages": 4,
    "samplingSteps": 120,
    "width": 1024}
I expected for for the request to be
Copy code
{
  "modelId": "pixel-art-xl-lora",
  "type": "character",
  "prompt": "a 3d rendering",
  "numOfImages": 4,
  "samplingSteps": 120,
  "width": 1024
}
not wrapped in input_request key I understand the alternative is not to use pydantic class and add each param explicitly in generate function (I prefer pydantic class) also, when using pil image as input, all request turns into files? it that on purpose?
j
Copy code
@bentoml.service
class MyService:
    @bentoml.api(input_spec=GenerateImageRequestBase)
    def generate(self, **params: t.Any) -> str:
        ...
This is how you would do it for your use case.
also, when using pil image as input, all request turns into files?
the request will become a 
multipart/form-data
request. I don't think it turns everything into file? @Frost Ming
a
Thanks! Not into file, I meant sent in -F
j
ah yes, -F is for 
form
a
@Jian Shen Yap another question, is it possible to catch error inside api code and control response example:
Copy code
try:
    model_params = map_params_to_pipeline(input_request)
except KeyError as exc:
    return JSONResponse(
        {"detail": f"Missing required parameter: {exc}"},
        status_code=400,
    )
f
@Amit Gelber yes, but instead, raise a bentoml exception instead:
Copy code
from bentoml.exceptions import BadInput
raise BadInput(f"Missing required parameter: {exc}")
The response body will be slightly different and not customizable though
a
thanks! works! another one, we are on a roll 🙂 how can start BentoML service in process for unitest? like https://fastapi.tiangolo.com/tutorial/testing/
@Frost Ming
f
That will be our next step and hopefully can be seen in the next release.
But actually you can directly instantiate the 
Service
and call it like a normal object.
Copy code
svc = MyService()
assert svc.predict(...) == expected
a
thanks! ill try it!
@Frost Ming what about testing the whole API? using requset.post(..) How can I initiate the class into a service? Thanks!
f
@Amit Gelber Upgrade to v1.2.7 and you can use the following api to get a test client
Copy code
import httpx
app = MyService.to_asgi(init=True)
client = httpx.AsyncClient(base_url="<http://testserver>", app=app)
response = await client.post("/predict", json=[[1,2,3,4]])
assert response.json() == [1]
a
Perfect! Thanks!
@Frost Ming help 🙂
Copy code
app = Txt2Img.to_asgi()
async with httpx.AsyncClient(base_url="<http://testserver>", app=app) as client:
    response = await client.post("/generate", json=detailed_data)
media_type = request.headers.get("Content-Type", "application/json") media_type = media_type.split(";")[0].strip() if self.is_main and media_type == "application/vnd.bentoml+pickle": # Disallow pickle media type for main service for security reasons raise BentoMLException( "Pickle media type is not allowed for main service", error_code=HTTPStatus.UNSUPPORTED_MEDIA_TYPE, ) method = self.service.apis[name] > func = getattr(self._service_instance, name) E AttributeError: 'NoneType' object has no attribute 'generate'
f
to_asgi(init=True)
🙌 1
Open in Slack
PreviousNext
Powered by