Hi, I'm trying to deploy my Bento service to Kubernetes and was hoping to get some pointers. 1. From what I understand, all I need to do is build and containerize my Bento, then use the created image for deployment. Is that correct? 2. Also, I'm using a Mac with Apple Silicon, but my models need to run on CUDA. What’s the correct way to build the image so that the service can utilize the GPU? Do I need to build it on a system with an Nvidia card? Right now, I’m passing the

--platform

flag with

amd64

or

mac

, because the image won't build with the default settings. 3. Lastly, after successfully building the image, I can't start it. It seems to be looking for the

service.py

file in the home directory, but it's actually located in the

src

subfolder. Here’s the error I’m getting:

Error: [serve] `serve` failed: Failed loading Bento from directory /home/bentoml/bento: Failed to import module "service": libGL.so.1: cannot open shared object file: No such file or directory

Any help would be appreciated! Thanks.

can you share the error logs when you containerize without the

--platform

target? I suppose you want to build on a linux target so that it deploys on k8.

Sorry, I was slightly off in my previous message. The error occurs after building the Bento without the

--platform

flag because of Nvidia-related packages.

uv pip install -r /home/bentoml/bento/env/python/requirements.lock.txt
0.352   × No solution found when resolving dependencies:
0.352   ╰─▶ Because nvidia-cublas-cu12==12.1.3.1 has no wheels with a matching
0.352       platform tag and you require nvidia-cublas-cu12==12.1.3.1, we can
0.352       conclude that your requirements are unsatisfiable.

So I used platform as suggested in the documentation. After that, how I build the Docker image doesn't seem to change anything, platform-wise. However, I still can't get the container to start. It's trying to find the code in the root directory instead of the

src

subdirectory(the error in first message). Am I missing something, or could this be a bug? I haven't seen anyone else report this issue.