Hi all, i am using the bentoml version==1.2.7 i am able to build the bento and all models are running in GPU, but when i try to run the docker image i am facing this error

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

Any inputs will help a lot, thanks

# ===========================================
#
# THIS IS A GENERATED DOCKERFILE. DO NOT EDIT
#
# ===========================================

# Block SETUP_BENTO_BASE_IMAGE
FROM nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu20.04 as base-container

ENV LANG=C.UTF-8

ENV LC_ALL=C.UTF-8

ENV PYTHONIOENCODING=UTF-8

ENV PYTHONUNBUFFERED=1

ENV NVIDIA_VISIBLE_DEVICES=0


USER root

ENV DEBIAN_FRONTEND=noninteractive
RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
RUN set -eux && \
    apt-get update -y && \
    apt-get install -q -y --no-install-recommends --allow-remove-essential \
        ca-certificates gnupg2 bash build-essential libblas-dev liblapack-dev gfortran
        
RUN apt install tmux && apt install nano

RUN \
    set -eux && \
    apt-get install -y --no-install-recommends --allow-remove-essential software-properties-common && \
    # add deadsnakes ppa to install python
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update -y && \
    apt-get install -y --no-install-recommends --allow-remove-essential curl python3.10 python3.10-dev python3.10-distutils

RUN ln -sf /usr/bin/python3.10 /usr/bin/python3 && \
    ln -sf /usr/bin/pip3.10 /usr/bin/pip3

RUN curl -O <https://bootstrap.pypa.io/get-pip.py> && \
    python3 get-pip.py && \
    rm -rf get-pip.py


# Block SETUP_BENTO_USER
ARG BENTO_USER=bentoml
ARG BENTO_USER_UID=1034
ARG BENTO_USER_GID=1034
RUN groupadd -g $BENTO_USER_GID -o $BENTO_USER && useradd -m -u $BENTO_USER_UID -g $BENTO_USER_GID -o -r $BENTO_USER

ARG BENTO_PATH=/home/bentoml/bento
ENV BENTO_PATH=$BENTO_PATH
ENV BENTOML_HOME=/home/bentoml/

RUN mkdir $BENTO_PATH && chown bentoml:bentoml $BENTO_PATH -R
WORKDIR $BENTO_PATH



# Block SETUP_BENTO_COMPONENTS
COPY --chown=bentoml:bentoml ./env/python ./env/python/
# install python packages with install.sh
RUN bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
COPY --chown=bentoml:bentoml . ./

# Block SETUP_BENTO_ENTRYPOINT
RUN rm -rf /var/lib/{apt,cache,log}
# Default port for BentoServer
EXPOSE 3000

# Expose Prometheus port
EXPOSE 3001

RUN chmod +x /home/bentoml/bento/env/docker/entrypoint.sh

USER bentoml

ENTRYPOINT [ "/home/bentoml/bento/env/docker/entrypoint.sh" ]

have you tried with the

--gpus

flag in docker?

--gpus=all

https://docs.docker.com/desktop/gpu/

Hey, i have solved this issue by updating nvidia-container-toolkit, thanks for the help