Hi guy I am new to bentoML and I have a simple question I am BentoML #ask-for-help

Hi guy, I am new to bentoML and I have a simple qu...

Leo Chen

05/07/2024, 4:29 PM

Hi guy, I am new to bentoML and I have a simple question. I am building a random forest model and trying to deployment on BentoCloud. When I do bentoml deploy . it kept in “deploying” status and the log show error message like this:

Copy code

bentoml.exceptions.NotFound: no Models with name 'tennis-predictor' exist in BentoML store <osfs '/home/bentoml/bento/models'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/_bentoml_sdk/service/factory.py", line 230, in __call__
    instance = self.inner()
  File "/home/bentoml/bento/src/service.py", line 16, in __init__
    self.model: bentoml.Model = bentoml.models.get("tennis-predictor:latest")
  File "/usr/local/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/usr/local/lib/python3.10/site-packages/bentoml/models.py", line 45, in get
    return _model_store.get(tag)
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/store.py", line 149, in get
    self._recreate_latest(_tag)
  File "/usr/local/lib/python3.10/site-packages/bentoml/_internal/store.py", line 120, in _recreate_latest
    raise NotFound(

Before I start deployment, the model is saved in the following python code, so Im pretty sure the model is saved in cloud.

Copy code

model_saved = bentoml.sklearn.save_model("tennis-predictor", rf_classifier)
    print(f"Model saved:{model_saved}")
    bentoml.models.push("tennis-predictor:latest")

Here is my service.py

Copy code

import bentoml
import numpy as np
import pandas as pd


@bentoml.service(
    resources={"cpu": "2"},
    traffic={"timeout": 10},
)

class TennisPrediction:
    def __init__(self) -> None:
        self.model = bentoml.sklearn.load_model('tennis-predictor:latest')
        # self.model: bentoml.Model = bentoml.models.get("tennis-predictor:latest")

    @bentoml.api
    def tennis_match_prediction(self, odd1: float, odd2: float) -> int:
        pre_data = {
            'Odd_1': [odd1],
            'Odd_2': [odd2],
        }
        pre_data = pd.DataFrame(pre_data)

        predict = self.model.predict(pre_data)[0]
        return predict

and my yaml file:

Copy code

service: "service:TennisPrediction"
labels:
  owner: leo
  project: gallery
include:
  - "*.py"
python:
  packages:
    - scikit-learn
    - pandas
    - numpy

Hope someone can help me with this, and I appreciate your amazing works.

Jian Shen Yap

05/09/2024, 9:08 AM

Check this doc out! https://docs.bentoml.com/en/latest/guides/model-store.html#retrieve-a-model you will have to either do the

load_model

as a class variable, or manually annotate the model dependencies in the yaml file

Leo Chen

05/09/2024, 12:00 PM

Thanks @Jian Shen Yap, in my case i can’t define model as class variable, so i decide to annotate model in yaml file. However, I get the following error:

Copy code

Error: [bentoml-cli] `deploy` failed: Failed to create deployment due to invalid configuration: request failed with status code 400: {"error":"create modelRepository model: ERROR: duplicate key value violates unique constraint \"uk_model_modelRepositoryId_version\" (SQLSTATE 23505)"}

my model annotation:

Copy code

models:
  - "tennis-predictor:latest"

service.py:

Copy code

class TennisPrediction:
    def __init__(self) -> None:
        model = bentoml.sklearn.load_model('tennis-predictor:latest')

Jian Shen Yap

05/09/2024, 12:02 PM

Hello, can I know which organization is this?

Leo Chen

05/09/2024, 12:03 PM

Im just a user learning to use BentoML lately, not representing any organization.

Jian Shen Yap

05/09/2024, 12:05 PM

Oh I meant your organization name for BentoCloud!

Leo Chen

05/09/2024, 12:08 PM

Ah sorry, Im very fresh for this. Is this the one you asking?

Jian Shen Yap

05/09/2024, 12:08 PM

ah yes!

Jian Shen Yap

05/09/2024, 12:09 PM

May i ask what is the commands that you run that trigger this error?

Copy code

Error: [bentoml-cli] `deploy` failed: Failed to create deployment due to invalid configuration: request failed with status code 400: {"error":"create modelRepository model: ERROR: duplicate key value violates unique constraint \"uk_model_modelRepositoryId_version\" (SQLSTATE 23505)"}

Leo Chen

05/09/2024, 12:10 PM

Copy code

bentoml deploy .

Jian Shen Yap

05/09/2024, 12:12 PM

This may be a bug, could you try deleting this model and see if the error still exist?

Leo Chen

05/09/2024, 12:16 PM

But if i delete this and deploy again, the service will not be able to load this model right? should i save another model on remote?

Leo Chen

05/09/2024, 12:21 PM

May i ask another question. When I annotate the model in yaml file like this, the service will load the model from BentoCloud instead of local, is that correct?

Jian Shen Yap

05/09/2024, 12:23 PM

should i save another model on remote?

You shouldn't need to! The

deploy

command will check if the model exist, if not then upload from the local.

the service will load the model from BentoCloud instead of local,

This will tell BentoCloud to pull the model into the Deployment, so that

bentoml.load

can find the model

Leo Chen

05/09/2024, 12:25 PM

So i try bentoml deploy . agin and still get the same issue. And i confirm models in my BentoCloud is empty

Copy code

Error: [bentoml-cli] `deploy` failed: Failed to create deployment due to invalid configuration: request failed with status code 400: {"error":"create modelRepository model: ERROR: duplicate key value violates unique constraint \"uk_model_modelRepositoryId_version\" (SQLSTATE 23505)"}

Jian Shen Yap

05/09/2024, 12:31 PM

I'm looking into this!

Leo Chen

05/09/2024, 12:33 PM

Thanks, Jian! Please let me know if there is any solution.

Jian Shen Yap

05/09/2024, 12:59 PM

I believe this is a bug on our side, we will fix it promptly

👍 1

Leo Chen

05/09/2024, 1:04 PM

Im wondering is there any workaround I can apply to load model in service instead of defining as a class variable?

Jian Shen Yap

05/09/2024, 1:05 PM

models:

- "tennis-predictor:latest"

^ this should solve the problem but i believe we have a bug in the pipeline. will let you know once its fixed!

Leo Chen

05/09/2024, 1:09 PM

Im still getting the same error when annotation like this🥲 Maybe there is any other issue in my yaml file?

Copy code

service: "service:TennisPrediction"
labels:
  owner: leo
  project: gallery
include:
  - "*.py"
models:
  - "tennis-predictor:latest"
python:
  packages:
    - scikit-learn
    - pandas
    - numpy

Jian Shen Yap

05/09/2024, 1:09 PM

No @Leo Chen, it is a bug from BentoML side. We will fix it and notify you prompty. Thanks for your patience

🙏 1

Nuwan

06/07/2024, 6:46 AM

is this fixed?

Jian Shen Yap

06/07/2024, 6:54 AM

hello! yes @Nuwan

👀 1

Nuwan

06/07/2024, 6:55 AM

I manually deleted all the deployment, bentos, and models in bentcloud and getting this error when trying to deploy again . 👀

Nuwan

06/07/2024, 6:56 AM

bentoml.exceptions.BentoMLException: Failed to create deployment due to invalid configuration: request failed with status code 400: {"error":"create modelRepository model: ERROR: duplicate key value violates unique constraint \"uk_model_modelRepositoryId_version\" (SQLSTATE 23505)"}

Nuwan

06/07/2024, 7:01 AM

It works after renaming the version tags. seems like it occurred when trying deploy again with same version tags. I used mlflow run_ids to automate the process.

Open in Slack

Previous Next