https://bentoml.com logo
Join Slack
Powered by
~Does `bentoml build --containerize .` no longer w...
# ask-for-help
a
Does 
bentoml build --containerize .
no longer work, especially with 
bentoml>=1.2.x
? Details in the 🧵~.~ Sorry, it seems I need to load the model (or at least call 
bentoml.models.get
not from the 
__init__
method but instead assign the result of that to a class variable. This wasn’t that clear to me from the documentation on the Services doc page.
This is using 
bentoml==1.2.12
. • train.py
Copy code
import bentoml
from sklearn import linear_model


if __name__ == "__main__":
    reg = linear_model.LinearRegression()
    reg.fit([[0, 0], [1, 1], [2, 2]], [0, 1, 2])

    print("coef: ", reg.coef_)
    bento_model = bentoml.sklearn.save_model("linear_reg_1_2", reg)
    print(f"Model saved: {bento_model}")

    # Test running inference with BentoML runner
    test_runner = bentoml.sklearn.get("linear_reg:latest").to_runner()
    test_runner.init_local()
    assert test_runner.predict.run([[1, 1]]) == reg.predict([[1, 1]])
• service.py
Copy code
import bentoml
import numpy as np

@bentoml.service(
    resources={"cpu": "2"},
    traffic={"timeout": 10},
)
class LinReg:
    def __init__(self) -> None:
        import joblib

        self.model_info = bentoml.models.get("linear_reg_1_2:latest")
        self.model = joblib.load(self.model_info.path_of("saved_model.pkl"))

    @bentoml.api
    def predict(self, input_arr: np.ndarray) -> np.ndarray:
        result = self.model.predict(input_arr)
        return result
Then if I build the Bento with 
bentoml build --container .
and try to run the resulting Docker container, I get an error saying the model isn’t in the container. Has something broken with the upgrade to 
1.2.x
?
Copy code
2024-04-22 14:10:52 Traceback (most recent call last):
2024-04-22 14:10:52   File "/usr/local/lib/python3.11/site-packages/_bentoml_sdk/service/factory.py", line 230, in __call__
2024-04-22 14:10:52     instance = self.inner()
2024-04-22 14:10:52                ^^^^^^^^^^^^
2024-04-22 14:10:52   File "/home/bentoml/bento/src/service.py", line 12, in __init__
2024-04-22 14:10:52     self.model_info = bentoml.models.get("linear_reg_1_2:latest")
2024-04-22 14:10:52                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-22 14:10:52   File "/usr/local/lib/python3.11/site-packages/simple_di/__init__.py", line 139, in _
2024-04-22 14:10:52     return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
2024-04-22 14:10:52            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-22 14:10:52   File "/usr/local/lib/python3.11/site-packages/bentoml/models.py", line 45, in get
2024-04-22 14:10:52     return _model_store.get(tag)
2024-04-22 14:10:52            ^^^^^^^^^^^^^^^^^^^^^
2024-04-22 14:10:52   File "/usr/local/lib/python3.11/site-packages/bentoml/_internal/store.py", line 149, in get
2024-04-22 14:10:52     self._recreate_latest(_tag)
2024-04-22 14:10:52   File "/usr/local/lib/python3.11/site-packages/bentoml/_internal/store.py", line 120, in _recreate_latest
2024-04-22 14:10:52     raise NotFound(
2024-04-22 14:10:52 bentoml.exceptions.NotFound: no Models with name 'linear_reg_1_2' exist in BentoML store <osfs '/home/bentoml/bento/models'>
Even including this in the 
benotfile.yaml
doesn’t seem to fix the issue:
Copy code
models:
  - "linear_reg_1_2:latest"
Sorry, it seems I need to instantiate something about the model and save the result of that as a class variable, not in the 
__init__
method. This wasn’t that clear to me from the docs.
j
Hello! Thanks for your feedback! https://docs.bentoml.com/en/latest/guides/model-store.html#retrieve-a-model Is this section unclear about having the model reference as a class variable?
a
Hi Jian, thanks for pointing me to that. I hadn’t seen that. I was only looking at examples from the services guide page which wasn’t as clear, since the first example uses an off-the-shelf example from 
transformers.pipeline
and the loading of that model is done in the 
__init__
method. The later example of the 
IrisClassifier
on that page does use a class-level variable, but it’s not clear from that page alone why that is necessary. But you’re right that the page you linked me to answers my question and has a helpful explanation as to why.
j
Got it! Thanks for the feedback! CC: @Sherlock Xu maybe we can put a link from Service page to the model store usage!
đź‘Ť 2
gratitude thank you 1
Open in Slack
PreviousNext
Powered by