👋 Hello! I'm migrating from

bentoml==1.11.*

to

1.3

and running into some issues when defining the input_spec when using pydantic models on the service. On

bentoml=1.11.*

:

@svc.api(input=create_input_spec(), output=JSON())
def predict(data: ModelInput) -> dict[str, Any]:

where

create_input_spec

:

from <http://bentoml.io|bentoml.io> import JSON
def create_input_spec():
    input_spec = JSON.from_sample(
        ModelInput(....) )
    return input_spec

while on

bentoml==1.3

@bentoml.service
class MyService:
    @bentoml.api(input_spec=ModelInput, output_spec=ModelOutput)
    def predict(self, data: ModelInput):

and when invoking

bentoml.build

I get this:

@bentoml.api(input=JSON(), output=JSON())
2024-08-20 09:59:28.225	
                       ^^^^
2024-08-20 09:59:28.225	
NameError: name 'JSON' is not defined

🤷‍♂️ - would be great with some migration guides on these releases

@Jean Poudroux, Im facing the same issue !! But if im right, the parameters in @bentoml.api() should be input_spec and output_spec but still it kept throwing error saying JSON should be a class type, when i pass pydantic base model. Hope there would be workaround to fix this.

@Jean Poudroux in 1.3 you would use the with

pydantic.Field

with default to provide an example value! https://docs.bentoml.com/en/latest/guides/iotypes.html#pydantic

@Jian Shen Yap, I did this and it worked but the problem is my input data structure has changed now. For ex, if i was passing { key1: value, key2:value } now its like { params: { key1: value, key2:value } } Could we avoid it by any chance ?

the second part of the doc section highlighted your usage.

from pydantic import BaseModel, Field
import typing as t
import bentoml

class AdsGenerationParams(BaseModel):
    prompt: str = Field(description="The prompt text")
    industry: str = Field(description="The industry the company belongs to")
    target_audience: str = Field(description="Target audience for the advertisement")
    temperature: float = Field(default=0.0, description="A sampling temperature between 0 and 2")

@bentoml.service
class AdsWriter:
    @bentoml.api(input_spec=AdsGenerationParams)
    def generate(self, **params: t.Any) -> str:

        # Access parameters from the request
        prompt = params['prompt']
        industry = params['industry']
        target_audience = params['target_audience']
        temperature = params['temperature']
        # Use the parameters in your Service logic
        # Implementation logic
        ...

I know, but im passing additional parameter bentoml context along with the params, so having a trouble. See below: These are the inputs for my api (self, data: SimulationMetadata, ctx: bentoml.Context): data i will get from the api