This message was deleted BentoML #ask-for-help

Join Slack

This message was deleted.

# ask-for-help

Slackbot

06/06/2023, 4:09 PM

This message was deleted.

Chaoyu

06/06/2023, 4:57 PM

Are you using the base_image option? Could you share your bentofile.yaml?

Matthieu Vanhoutte

06/06/2023, 7:34 PM

I am not using the base image option:

Copy code

service: "bentoml_gpu_ct2_service:cfct5s" # Same as the argument passed to `bentoml serve`
labels:
  owner: tinycoaching-ml-team
  stage: dev
include:
  - "*.py" # A pattern for matching which files to include in the bento
exclude:
  - "__pycache__/"
  - "transformers_save_model.py"
python:
  packages: # Additional pip packages required by the service
    - "transformers==4.29.2"
    - "tokenizers==0.13.3"
    - "sentencepiece==0.1.99"
    - "sacremoses==0.0.53"
    - "pydantic==1.10.8"
    - "six==1.16.0"
    - "websocket-client==1.5.2"
    - "python-dateutil==2.8.2"
    - "tzlocal==5.0.1"
    - "attrs==23.1.0"
    - "protobuf==3.20.2"
    - "ctranslate2==3.14.0"
    - "starlette==0.26.1"
docker:
  distro: debian
  python_version: "3.8.16"
  cuda_version: "11.6"

This generates in the Dockerfile

FROM nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04 as base-debian

which is not the version I want but

nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu18.04 as base-debian

So I modified the Dockerfile as explained above, which functioned with previous version of bentoML (v.1.0.7). But with version v1.0.21, whatever I specify in the Dockerfile, it downloads

nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04

Any idead @Chaoyu?

Chaoyu

06/06/2023, 7:40 PM

usually we don’t recommend modifying files in Bento because we treat it as immutable by design, and there must be some internal changes affecting this behavior 🤔 could you try using the base_image or docker template option instead?

Chaoyu

06/06/2023, 7:40 PM

https://docs.bentoml.org/en/latest/guides/containerization.html

Chaoyu

06/06/2023, 7:42 PM

I think this is broken because we introduced the

--enable-features

flag,

bentoml containerize

now will regenerate Dockerfile based on user provided arguments

Chaoyu

06/06/2023, 7:44 PM

however, if you are comfortable working with the lower-level of BentoML internals, you could also use

docker build

command and provide the corresponding Dockerfile path and set build context to the bento directory - this is a bit hacky and rely on BentoML’s internal implementation details. But hopefully can be a quick short-term fix for your case.

Matthieu Vanhoutte

06/07/2023, 7:50 AM

Thanks @Chaoyu your short-term fix works perfectly! I will try to use base_image in the future.

👍 1

Open in Slack

Previous Next