*What is the format of `JSON` data for inference A...
# ask-for-help
y
What is the format of
JSON
data for inference API when using model that has input-schema?
✅ 1
Hello, I trained scikit-learn model using
diabetes toy-dataset
based on
pandas.dataframe
not
numpy.array
object of dataset. So, my model has input-schema(column name of features). And I run bento serving server for this model, and I call inference API with below json data, I encountered error with input shcema. • my bento-ml serving source code
Copy code
import bentoml
import os


import numpy as np
import pandas as pd 


bento_svc_name = os.getenv("BENTOML_SVC_NAME", "tabular-diabetes-regressor-v2-bento-svc")
bento_model_name = os.getenv("BENTOML_MODEL_NAME", "tabular-diabetes-regressor-v2-bento-model")  # trained with `pandas.dataframe` not `numpy.array`!!
bento_model = bentoml.mlflow.get(f"{bento_model_name}:latest")
bento_legacy_runner = bento_model.to_runner()
bento_runner_svc = bentoml.runner_service(runner=bento_legacy_runner)


@bentoml.service(
    traffic={"timeout": 10},
    resources={"cpu": "2"},
    name=bento_svc_name
)
class DiabetesRegressor:
    runner = bentoml.depends(bento_runner_svc)
    
    @bentoml.api
    def predict(self, x):
        y = self.runner.predict(x)
        return y
• json data
Copy code
{
  "x": [
    [
      1,
      2,
      3,
      4,
      5,
      6,
      7,
      8,
      9,
      10
    ]
  ]
}
• error message
Copy code
Traceback (most recent call last):
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 418, in api_endpoint_wrapper
    resp = await self.api_endpoint(name, request)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 508, in api_endpoint
    output = await self._to_thread(func, *input_args, **input_params)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 363, in _to_thread
    output = await anyio.to_thread.run_sync(func, limiter=self._limiter)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/bentoml/_internal/runner/runnable.py", line 155, in method
    return self.func(obj, *args, **kwargs)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/bentoml/_internal/frameworks/mlflow.py", line 260, in predict
    return self.model.predict(input_data)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py", line 714, in predict
    data, params = self._validate_prediction_input(data, params)
  File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py", line 668, in _validate_prediction_input
    raise MlflowException.invalid_parameter_value(
mlflow.exceptions.MlflowException: Failed to enforce schema of data '[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0]]' with schema '['age': double (required), 'sex': double (required), 'bmi': double (required), 'bp': double (required), 's1': double (required), 's2': double (required), 's3': double (required), 's4': double (required), 's5': double (required), 's6': double (required)]'. Error: Model is missing inputs ['age', 'sex', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']. Note that there were extra inputs: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
To test other case that doesn't have input-schema model(trained with
numpy.array
), when I checked the case and the same json data format above, this case is 200 status. How to formatting json data for trained model with input-schema? plz help me!
j
Hi you can use a pydantic model in the type annotation to specify the input format! https://docs.bentoml.com/en/latest/guides/iotypes.html
👍 1
y
@Jian Shen Yap thanks for reply! I couldnt find your attached docs.. after i read it, I resolved my case!