Hi bento community Could anyone help me with this error I ve BentoML #ask-for-help

Hi bento community, Could anyone help me with this...

Serine Daouk

10/15/2024, 5:20 AM

Hi bento community, Could anyone help me with this error ? I've previously successfully deployed my bento as a containerized service on my machine to serve predictions. I use

bentoml build --containerize

and get no error. My image and container (once I start it with docker run ..) are started successfully, but when I check my container's logs I get this:

Copy code

2024-10-14T10:19:46+0000 [ERROR] [entry_service:MulticlassClassifier:1] Initializing service error
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/_bentoml_sdk/service/factory.py", line 251, in __call__
    instance = self.inner()
               ^^^^^^^^^^^^
  File "/home/bentoml/bento/src/multiclass_service.py", line 15, in __init__
    self.model = bentoml.mlflow.load_model(self.bento_model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bentoml/_internal/frameworks/mlflow.py", line 95, in load_model
    return mlflow.pyfunc.load_model(bento_model.path_of(MLFLOW_MODEL_FOLDER))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 871, in load_model
    raise e
  File "/usr/local/lib/python3.11/site-packages/mlflow/pyfunc/__init__.py", line 856, in load_model
    model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 468, in _load_pyfunc
    context, python_model, signature = _load_context_model_and_signature(model_path, model_config)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mlflow/pyfunc/model.py", line 450, in _load_context_model_and_signature
    python_model = cloudpickle.load(f)
                   ^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'src'

What could be the cause ? I don't know which

src

file it's talking about.

Chaoyu

10/15/2024, 5:49 AM

Could you share a bit more about your project setup and steps to reproduce?

Serine Daouk

10/15/2024, 6:11 AM

Hey Chaoyu, sure here's what I'm running: So I have a model stored in my mlflow registry, I import it this with the following

Copy code

model_name = "multiclass_classifier"
model_uri = f"models:/{model_name}/1"
bentoml.mlflow.import_model(model_name, model_uri)

into my local bento store (a simple bentoml directory on my server). I then

cd

into the directory where I have my

bentofile.yaml

and

multiclass_service.py

files and from within run the

bentoml build --containerize

to build a bento. multiclass_service.py:

Copy code

import bentoml
import numpy as np


@bentoml.service(http={"port": 3004}, worker=2)
class MulticlassClassifier:
    bento_model = bentoml.models.get("multiclass_classifier:latest")

    def __init__(self):
        self.model = bentoml.mlflow.load_model(self.bento_model)

    @bentoml.api
    def predict(self, input_data: np.ndarray) -> np.ndarray:
        return self.model.predict(input_data)

bentofile.yaml:

Copy code

service: "multiclass_service:MulticlassClassifier"
models:
  - "multiclass_classifier:latest"
include:
  - "multiclass_service.py"
python:
  packages:
  - numpy==1.26.4
  - mlflow==2.11.3
  - xgboost==2.1.1
  - scikit-learn==1.4.1.post1
  - lz4==4.3.2
  - pydantic==2.7.4
  - psutil==5.9.8
  - scipy==1.12.0
  - python-dotenv==1.0.1

I check to see if a new docker image is created (it is). Finally I run d`ocker run -d -p 3004:3004 multiclass_classifier:tag serve` and my docker container is up. I've done this before but using the mlflow run uri (runs:/...) for my model_uri instead of (models:/...) but now I'm loading my model from my registry and I can see it in my bentoml model store and as a bento (the build was also successful). That's why I'm trying to look for what could be causing this module not found error.

Chaoyu

10/15/2024, 6:29 AM

Could you try running just the

model = bentoml.mlflow.load_model(self.bento_model)

line and see if it’s an issue with the model?

Serine Daouk

10/15/2024, 6:33 AM

It runs successfully and also returns the corresponding

run_id

of the registered mlflow pyfunc model that we saved to bento. So the resulting model is of type

<class 'mlflow.pyfunc.PyFuncModel'>

Serine Daouk

10/15/2024, 6:33 AM

No errors there

Serine Daouk

10/15/2024, 6:52 AM

Hey Chaoyu, I think I've got a lead. The issue might be coming from my model training phase, it isn't mlflow nor bentoML related. I'll keep looking there to see what might be the issue at core causing it. Will update this thread for any new findings. Thanks again for the help 👍🏻

👍 2

Serine Daouk

10/17/2024, 10:18 AM

Hello again 👋🏻 So here's what was happening: This is an approximated example of my file system:

Copy code

|
|_ bento/multiclass_classifier/bentofile.yaml
|_ bento/multiclass_classifier/multiclass_service.py
|
|_ notebooks/my_notebook.ipynb
|
|_ src/train.py
|_ utils/..

I was previously (1) training my models in a

my_notebook.ipynb

then (2) having it logged onto my remote mlflow store and then (3) loaded the artifact to my bento store [side-note: why is this required ?] and (4) building my service by running '`bentoml build --containerize`' from within the

/bento/multiclass_classifier

directory and starting up my service (

docker run ..

) to handle incoming requests. Everything worked just fine. Now that I've moved my training scripts into

/src/train.py

I was still running the

bento build ...

command from inside the

/bento/multiclass_classifier

directory but I didn't have

src/

included inside my

bentofile.yaml

I hadn't realized I needed to mention from where my model was being trained from because I assumed that once the model artifact (my

mlflow.pyfunc.PyFuncModel

model) got placed into the bento store that it just gets picked up from there when it had to run the

bentoml.mlflow.load_model(...)

command of my service to fetch it for inference. Finally after some digging I finally figured it out and currently placed both of my

bentofile.yaml

and

multiclass_service.py

files on the same level as

src

and made sure to have

src/

included in my

bentofile.yaml

include section. Now the service is running and listening waiting for inference requests. Please let me know if you have any additional tips or comments about the matter, and also thank you for holding this ask-for-help section, it really shows your support to end-users using this framework. Thank again!

Open in Slack

Previous Next