YoungHun Jo
06/06/2024, 6:20 AMJSON data for inference API when using model that has input-schema?YoungHun Jo
06/06/2024, 6:20 AMdiabetes toy-dataset based on pandas.dataframe not numpy.array object of dataset. So, my model has input-schema(column name of features). And I run bento serving server for this model, and I call inference API with below json data, I encountered error with input shcema.
• my bento-ml serving source code
import bentoml
import os
import numpy as np
import pandas as pd
bento_svc_name = os.getenv("BENTOML_SVC_NAME", "tabular-diabetes-regressor-v2-bento-svc")
bento_model_name = os.getenv("BENTOML_MODEL_NAME", "tabular-diabetes-regressor-v2-bento-model") # trained with `pandas.dataframe` not `numpy.array`!!
bento_model = bentoml.mlflow.get(f"{bento_model_name}:latest")
bento_legacy_runner = bento_model.to_runner()
bento_runner_svc = bentoml.runner_service(runner=bento_legacy_runner)
@bentoml.service(
traffic={"timeout": 10},
resources={"cpu": "2"},
name=bento_svc_name
)
class DiabetesRegressor:
runner = bentoml.depends(bento_runner_svc)
@bentoml.api
def predict(self, x):
y = self.runner.predict(x)
return y
• json data
{
"x": [
[
1,
2,
3,
4,
5,
6,
7,
8,
9,
10
]
]
}
• error message
Traceback (most recent call last):
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 418, in api_endpoint_wrapper
resp = await self.api_endpoint(name, request)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 508, in api_endpoint
output = await self._to_thread(func, *input_args, **input_params)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 363, in _to_thread
output = await anyio.to_thread.run_sync(func, limiter=self._limiter)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
return await future
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
result = context.run(func, *args)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/bentoml/_internal/runner/runnable.py", line 155, in method
return self.func(obj, *args, **kwargs)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/bentoml/_internal/frameworks/mlflow.py", line 260, in predict
return self.model.predict(input_data)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py", line 714, in predict
data, params = self._validate_prediction_input(data, params)
File "/Users/zedd.ai/.pyenv/versions/home-mlops/lib/python3.10/site-packages/mlflow/pyfunc/__init__.py", line 668, in _validate_prediction_input
raise MlflowException.invalid_parameter_value(
mlflow.exceptions.MlflowException: Failed to enforce schema of data '[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0]]' with schema '['age': double (required), 'sex': double (required), 'bmi': double (required), 'bp': double (required), 's1': double (required), 's2': double (required), 's3': double (required), 's4': double (required), 's5': double (required), 's6': double (required)]'. Error: Model is missing inputs ['age', 'sex', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']. Note that there were extra inputs: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
To test other case that doesn't have input-schema model(trained with numpy.array ), when I checked the case and the same json data format above, this case is 200 status.
How to formatting json data for trained model with input-schema? plz help me!Jian Shen Yap
06/07/2024, 1:58 AMYoungHun Jo
06/07/2024, 5:27 AM