This message was deleted.
# ask-for-help
s
This message was deleted.
c
Are you using the base_image option? Could you share your bentofile.yaml?
m
I am not using the base image option:
Copy code
service: "bentoml_gpu_ct2_service:cfct5s" # Same as the argument passed to `bentoml serve`
labels:
  owner: tinycoaching-ml-team
  stage: dev
include:
  - "*.py" # A pattern for matching which files to include in the bento
exclude:
  - "__pycache__/"
  - "transformers_save_model.py"
python:
  packages: # Additional pip packages required by the service
    - "transformers==4.29.2"
    - "tokenizers==0.13.3"
    - "sentencepiece==0.1.99"
    - "sacremoses==0.0.53"
    - "pydantic==1.10.8"
    - "six==1.16.0"
    - "websocket-client==1.5.2"
    - "python-dateutil==2.8.2"
    - "tzlocal==5.0.1"
    - "attrs==23.1.0"
    - "protobuf==3.20.2"
    - "ctranslate2==3.14.0"
    - "starlette==0.26.1"
docker:
  distro: debian
  python_version: "3.8.16"
  cuda_version: "11.6"
This generates in the Dockerfile
FROM nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04 as base-debian
which is not the version I want but
nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu18.04 as base-debian
So I modified the Dockerfile as explained above, which functioned with previous version of bentoML (v.1.0.7). But with version v1.0.21, whatever I specify in the Dockerfile, it downloads
nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04
Any idead @Chaoyu?
c
usually we don’t recommend modifying files in Bento because we treat it as immutable by design, and there must be some internal changes affecting this behavior 🤔 could you try using the base_image or docker template option instead?
I think this is broken because we introduced the
--enable-features
flag,
bentoml containerize
now will regenerate Dockerfile based on user provided arguments
however, if you are comfortable working with the lower-level of BentoML internals, you could also use
docker build
command and provide the corresponding Dockerfile path and set build context to the bento directory - this is a bit hacky and rely on BentoML’s internal implementation details. But hopefully can be a quick short-term fix for your case.
m
Thanks @Chaoyu your short-term fix works perfectly! I will try to use base_image in the future.
👍 1