My service runs multiple mini models, so I either need some generic way of adding many mini models to the bento or otherwise I need to wrap them in a single model

could you share your example code for us to have a better understanding of what you are trying to achieve?

Instead of this:

@bentoml.service
class Service:
    model1 = BentoModel(env_config.model1_tag)
    model2 = BentoModel(env_config.model2_tag)
    model3 = BentoModel(env_config.model3_tag)
    ...
    def __init__(self):
        # prepare model1
        self.prep_model(model1)
        # etc

    @bentoml.api
    def predict(self, input: Input) -> Output:
        if input.model == "model1":
            return model1(input)
        if input.model == "model2":
            return model2(input)
        ...

Id like something like this:

@bentoml.service
class Service:
    def __init__(self):
        self.models = self.prep_models()

    @bentoml.api
    def predict(self, input: Input) -> Output:
        model = self.models[input.model]
        return model.predict(input)

I'm not 100% sure. But we have many frontends with forms, to which I am supplying an anomaly detection service. Each form has its own VAE model. Currently it's only 3 models, of around 20-40 kb each, inferencing in about a few ms, but it will grow over time, so I'm looking for a more generic approach.

How bentoml determines what model to include is scanning class attribute the the type of BentoModel. though you can explicitly include the model in the bentofile.yaml, which means you would need to programmtically generate the bentofile!

This is still possible:

@bentoml.service
class Service:
    model1 = BentoModel(env_config.model1_tag)
    model2 = BentoModel(env_config.model2_tag)
    model3 = BentoModel(env_config.model3_tag)
    
    def __init__(self):
        self.models = self.prep_models()

    @bentoml.api
    def predict(self, input: Input) -> Output:
        model = self.models[input.model]
        return model.predict(input)

As jianshen said, the models must be a unique class attribute on each, this is required for the bentoml to collect all used models