I noticed a strange issue: when my instance count is 0, the first request I send always returns “input parameter is missing.” However, once there is at least one instance, this problem does not occur. After the first request is sent, the number of instances will expand (from 0 to 1). The second request is sent immediately after the completion of the first request (at this point, the instance has not scaled down and there is still one instance).

cc @Frost Ming

I failed to reproduce using a simple service:

from typing import Annotated

import bentoml
from bentoml.validators import ContentType
from PIL import Image
from pydantic import Field


@bentoml.service(image=bentoml.images.Image().python_packages("pillow"))
class MyService:
    @bentoml.api
    def generate(
        self,
        image: Annotated[Image.Image, ContentType("image/png")] = Field(
            description="PNG image"
        ),
    ) -> str:
        return f"Received image with size: {image.size} and mode: {image.mode}"

The input signature is exactly the same as yours

@Frost Ming I found that this only appears in the services of my account. I tried to deploy the same service using the company’s account (the code is the same, only the specifications are different: mine is gpu.t4, and the company’s is gpu.l4) and there was no issue.