Dear everyone, hi. I want to run a LLM model of the default repository of OpenLLM with a Kubernetes cluster. At least, right now, the following is wrong: https://cheatsheet.md/llm-leaderboard/openllm Since OpenLlm don't has the build command: ubuntu@ip-10-0-21-129:~$ openllm build gemma3 --model-id gemma3/gemma3:1b Usage: openllm [OPTIONS] COMMAND [ARGS]... Try 'openllm -h' for help. ╭─ Error ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ No such command 'build'. │ ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ How should I do? I'm glad if you answer it.

you can create a container that has the entrypoint command of

openllm serve xxx

or for more optimized deployment, you can tryout deploying them in bentocloud. https://github.com/bentoml/OpenLLM

@Jian Shen Yap Thanks for answering. Well, does the command create the container image? I tried the serve command but I seem that the image wasn't created. I want to create the container image for deploying to a Kubernetes cluster.

we also have a vllm bento https://github.com/bentoml/BentoVLLM