BentoML

Hello everyone. I am using deepspeed library for improve the speed inference of model. Do you have any tips for installing nvcc compiler dependencies on when we build the container file ?

is there any guidance for usage of

`nvidia/cuda:11.8.0-devel-runtime-ubuntu20.04 as base-container`

Instead of:
`nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu20.04 as base-container`

?

I believe you just need to set `cuda_version: "11.8.0"` in the `bentofile.yaml`

See <https://docs.bentoml.com/en/latest/guides/build-options.html#docker>

When I do this it not install the nvidia-toolkit which I need for nvcc compiler. The reason why I need this is: I want to use the deepspeed library for speedup for inference.