Hello, I have two questions: 1. Does bento support model updates? Is it implemented directly by modifying service.py? 2. Will bento download the model to the image and package it? Is there a difference between the model download logic of llm and non-llm?

Does bento support model updates

What do you mean by model updates? We have a Model store where you can save your model https://docs.bentoml.com/en/latest/build-with-bentoml/model-loading-and-management.html and version them with tags.

Will bento download the model to the image and package it?

bentoml containerize packs the model into the image, excepts

HuggingFaceModel

https://docs.bentoml.com/en/latest/build-with-bentoml/model-loading-and-management.html#load-a-model

Is there a difference between the model download logic of llm and non-llm

not in BentoML. In BentoCloud we have implmented technique to speed up LLM downloads

@Jian Shen Yap Hi, can I understand it this way? If I use bento apply to a model deepseek, and then want to update the model to qwen, I need to update service.py and then execute bento apply. In fact, this apply action corresponds to a configuration update action of a deployment object.

Yes. Another way is that you can use an environment variable such that you can inject your model without needing to change code