Hi there, I am trying to deploy my model on BentoCloud, but I keep getting this error on BentoCloud when I performed prediction by bentoml.SyncHTTPClient.

pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'numpy.ndarray'>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 561, in api_endpoint_wrapper
    resp = await self.api_endpoint(name, request)
  File "/opt/conda/lib/python3.10/site-packages/_bentoml_impl/server/app.py", line 660, in api_endpoint
    response = await method.output_spec.to_http_response(output, serde)
  File "/opt/conda/lib/python3.10/site-packages/_bentoml_sdk/io_models.py", line 259, in to_http_response
    payload = serde.serialize_model(ins)
  File "/opt/conda/lib/python3.10/site-packages/_bentoml_impl/serde.py", line 150, in serialize_model
    model.model_dump_json(
  File "/opt/conda/lib/python3.10/site-packages/pydantic/main.py", line 415, in model_dump_json
    return self.__pydantic_serializer__.to_json(

Could someone explain what this means? I am confused since my output is not a numpy array, it is a list with json format elements in it.

@bentoml.service(
    resources={"cpu": "2"},
)
class BentoService:

    bento_model_ref = bentoml.models.get("classification-model:latest")
    def __init__(self):
        self.model = bentoml.mlflow.load_model(self.bento_model_ref)

    @bentoml.api
    def predict(self, test_data: str) -> list:
        test_data_df = pd.read_json(test_data, orient="split")
        predictions = self.model.predict(test_data_df)

        return predictions

Hi could you share the link to the deployment itself?

Hi @Jian Shen Yap https://bento-service-test-ue0e-617b0545.mt-guc1.bentoml.ai This is the deployment link

Does it work with a curl or normal http request?

I just try with normal request, it shows the same error

does it works locally? Do you mind sharing an example request data that will throw this error

It does not work locally, shows the same error

Hi

Hi @Jiang I got this error after adding tolist()

AttributeError: 'list' object has no attribute 'tolist'

I believe the prediction output is a list.

Then I guess it is a list of numpy ndarray. We shall call tolist for each item of list recursively. Do you need an example code?

Thanks for your assistance. The issue was that the json contained arrays, and JSON cannot serialize arrays directly. I converted all arrays to lists, and that resolved the issue.