BentoML

Hi you need to include the cuda dependencies in yaml file:

for example:
`service: "service.py:svc"`
`include:`
  `- "service.py"`
  `- "requirements.txt"`
  `- "models/v1_4_fp16"`
  `- "configuration.yaml"`
`python:`
  `packages:`
    `- torch`
    `- transformers`
    `- diffusers`
    `- ftfy`
`docker:`
    `distro: debian`
    `cuda_version: "11.6.2"`
    `env:`
      `BENTOML_CONFIG: "src/configuration.yaml"`

I have tried that, it doesn't work, all that does is use a docker base image that has cuda. I'm already using an image that has cuda so it changes nothing........

can you paste your bentoml file here?

are you saying tha you are using another image (that has cuda) as base image? in this case look at <https://docs.bentoml.org/en/latest/guides/containerization.html#custom-base-image>

it says that if custom base image is provided then user must take care of other things like python version etc. Also it ignores the cuda realted things in bento file.