Hi! Thank you for the new changes in Bentoml model...
# ask-for-help
a
Hi! Thank you for the new changes in Bentoml model store. it enables many more use cases. 2 questions: 1. when will metadata be deprecated from the bentoml models store? will the store have a similar feature? 2. will you support in the future the "coupled" framework integration with runners? meaning, bentoml diffusers. to runner.. will be supported in future? should we migrate to the new SDK which does support to runner feature? ( with all runner code) ?
c
For 1, do you have an use case that needs the metadata field? Curious to learn more.
We may keep it for future model store integrations, although I believe currently it’s not an user facing API
2. Yes we are thinking about that to simplify the migration process!
a
@Chaoyu we use it to store LoRa for diffusion models. each LoRa has attributes like the type of use case. when we execute bentoml.models.list() we are filtering on the type of use case according to user request
c
How do you save the Lora layers with model store today?
Btw the metadata field is always there in the model store, we don’t plan on deprecating it
a
using the new sdk:
Copy code
with bentoml.models.create(
        name='lora-use-case',
        metadata={"studio": "sm", "type": "lora","use-case":"background", "height": 1024, "width": 1024}
) as model_ref:
    # Copy the entire model directory to the BentoML Model Store
    shutil.copytree(local_model_dir, model_ref.path, dirs_exist_ok=True)
    print(f"Model saved: {model_ref}")
later on we created a service which execute bentoml.models.list() and filter on type and use case in the old SDK i couldn't save it