https://bentoml.com logo
Join Slack
Powered by
Hi, I have created a docker container using this s...
# ask-for-help
m
Hi, I have created a docker container using this script files:
Copy code
### model creation
import joblib
import bentoml
import pickle

pickle_file = "breast_cancer_model.pkl"

with open(pickle_file, "rb") as f:
    model = pickle.load(f)

with bentoml.models.create("breast_cancer_model") as bento_model:
    joblib.dump(model, bento_model.path_of("model.pkl"))

print(f"Model saved: {bento_model}")

### model service
import bentoml
import numpy as np
from pydantic import Field
import joblib

@bentoml.service(
    resources={
        "cpu": "1",
        "memory": "2Gi",
    },
)
class BreastCancerClassifier:
    bcc_model = bentoml.models.BentoModel("breast_cancer_model:latest")

    def __init__(self) -> None:
        self.model = joblib.load(self.bcc_model.path_of("model.pkl"))

    @bentoml.api
    def predict(self, input: np.ndarray = Field(examples=[[7.76, 47.92, 181, 0,  9.456,  59.16, 268.6, 0]])) -> str:
        result = self.model.predict(input)
        return 'M' if result[0] else 'B'

### bentoml config
# <https://docs.bentoml.com/en/latest/reference/bentoml/bento-build-options.html>
service: "bentoApi.py:BreastCancerClassifier"
labels:
    owner: Glorfindel
    project: BreastCancerClassifier
include:
    - "bentoApi.py"
python:
    lock_packages: false
    requirements_txt: "requirements.txt"
    no_index: false
    trusted_host:
        - "pypi.python.org"
        - "pypi.org"
        - "files.pythonhosted.org"
docker:
  distro: debian
  python_version: "3.11"
When I try to launch docker run I received this error:
Copy code
docker run -p 3000:3000 cr/breast-cancer-service:breast-cancer-model-1
2025-04-01T09:31:35+0000 [INFO] [cli] Service loaded from Bento directory: <Service name='BreastCancerClassifier'>
2025-04-01T09:31:35+0000 [INFO] [cli] Service loaded from Bento directory: <Service name='BreastCancerClassifier'>
Error: [serve] `serve` failed: No cloud context default found
Can you help me?
j
@Frost Ming could this be a bug where it requires login of cloud context when using 
BentoModel
m
Why should I authenticate? I have a k8s deployment and I can pull the image using pull secret.. The problem is when the container starts execution.. I can overwrite the entry point command: is there a command to avoid this login? Thanks
I try to overwrite the default command:
Copy code
spec:
  containers:
    - args:
        - . /app/.venv/bin/activate && bentoml serve
      command:
        - /bin/bash
        - '-c'
In this way I reproduced the error
Using --debug parameter this is the stack trace:
Copy code
2025-04-01T12:36:51+0000 [DEBUG] [cli] Loading bento "breast_cancer_classifier:pp5fduqo46bt47so" from directory: /home/bentoml/bento
2025-04-01T12:36:51+0000 [DEBUG] [cli] Importing service "bentoApi:BreastCancerClassifier" from working dir: "/home/bentoml/bento/src"
2025-04-01T12:36:51+0000 [INFO] [cli] Service loaded from Bento directory: <Service name='BreastCancerClassifier'>
2025-04-01T12:36:51+0000 [DEBUG] [cli] Loading bento "breast_cancer_classifier:pp5fduqo46bt47so" from directory: /home/bentoml/bento
2025-04-01T12:36:51+0000 [DEBUG] [cli] Importing service "bentoApi:BreastCancerClassifier" from working dir: "/home/bentoml/bento/src"
2025-04-01T12:36:51+0000 [INFO] [cli] Service loaded from Bento directory: <Service name='BreastCancerClassifier'>
2025-04-01T12:36:51+0000 [DEBUG] [cli] GPU not detected. Unable to initialize pynvml lib.
2025-04-01T12:36:51+0000 [DEBUG] [cli] Tracking Error: No cloud context default found
Error: [serve] `serve` failed: No cloud context default found
Traceback (most recent call last):
  File "/app/.venv/bin/bentoml", line 10, in <module>
    sys.exit(cli())
             ^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml_cli/utils.py", line 369, in wrapper
    raise err from None
  File "/app/.venv/lib/python3.11/site-packages/bentoml_cli/utils.py", line 364, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml_cli/utils.py", line 335, in wrapper
    return_value = func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml_cli/env_manager.py", line 126, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml_cli/serve.py", line 313, in serve
    serve_http(
  File "/app/.venv/lib/python3.11/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/_bentoml_impl/server/serving.py", line 228, in serve_http
    server_on_deployment(svc)
  File "/app/.venv/lib/python3.11/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/_bentoml_impl/server/serving.py", line 141, in server_on_deployment
    model.to_model().resolve()
  File "/app/.venv/lib/python3.11/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/_bentoml_sdk/models/base.py", line 136, in resolve
    cloud_client = BentoMLContainer.bentocloud_client.get()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/simple_di/__init__.py", line 72, in get
    return self._provide()
           ^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/simple_di/providers.py", line 124, in _provide
    value = super()._provide()
            ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/simple_di/providers.py", line 103, in _provide
    return inject(self._func)(
           ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml/_internal/configuration/containers.py", line 510, in bentocloud_client
    return BentoCloudClient.for_context(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml/_internal/cloud/__init__.py", line 67, in for_context
    ctx = cfg.get_context(context)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.11/site-packages/bentoml/_internal/cloud/config.py", line 84, in get_context
    raise CloudRESTApiClientError(
bentoml.exceptions.CloudRESTApiClientError: No cloud context default found
f
what version of bentoml
m
1.4.7
f
oh, didn’t you use 
bentoml build
and 
bentoml containerize
to get the image?
m
more or less.. I have a tekton task where in a step I build the bundle:
Copy code
bentoml build
echo "------------------------------ Complete bentoml bundle"
CLASS_NAME=$(ls -t ~/bentoml/bentos | head -1)
CLASS_NAME=${CLASS_NAME%/}
echo "CLASS_NAME: $CLASS_NAME"
SAVED_PATH=$(bentoml get $CLASS_NAME:latest -o path)
mkdir /workspace/build-image
echo "bentoml bundle on $SAVED_PATH" 
echo "------------------------------ Prepare data"   
cp $SAVED_PATH/env/docker/Dockerfile $SAVED_PATH/Dockerfile
cd $SAVED_PATH
tar -cvzf "$CLASS_NAME.tar.gz" .
cp $CLASS_NAME.tar.gz /workspace/build-image/$CLASS_NAME.tar.gz
and then I use buildah to create image:
Copy code
cd /workspace/build-image
CONTEXT=$(cat /workspace/build-image/CONTEXT)
echo "CONTEXT: $CONTEXT"
mkdir $BENTOML_BUILD_DIR
tar -xzf $CONTEXT -C $BENTOML_BUILD_DIR
IMAGE=$BENTOML_CR_REPO/$(params.image-id)
echo "IMAGE: $IMAGE"
echo "------------------------------ Start Buildah executor with bentoml image"
CERT_DIR_FLAG="--cert-dir /root/ssl/certs"
buildah ${CERT_DIR_FLAG} bud --no-cache --file $BENTOML_BUILD_DIR/Dockerfile --storage-driver vfs --format docker --tag $IMAGE $BENTOML_BUILD_DIR 
buildah ${CERT_DIR_FLAG} push --authfile /root/push-destination-auth --storage-driver vfs --digestfile /tmp/image-digest $IMAGE
cat /tmp/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -e -n "\n$IMAGE\n" | tee $(results.IMAGE_URL.path)
until version bentoml==1.0.16 I have not any problems
after upgrade yes
f
oh, from 1.2 the models are no longer bundled with the bento but instead pulled when running 
bentoml containerize
you may need to adjust your workflow
m
bad news
is there no other way?
f
i don't think it's that bad. you can even copy the models into the bento yourself
m
no no it's wonderful !! but my context is different: my flow is executed in a container and I must create a container using 
build containerize
.. container in a container.. I can specify buildah/kaniko executor instead docker for example?
f
that is what i mean: to copy the models into the bento folder and continue your workflow to build a container from that
Open in Slack
PreviousNext
Powered by