Hello, I’m new to bentoml, but I find it very interesting and thank you all for such a great project! I was wondering if it’s possible to load a service at runtime. Let’s say I have multiple models in a single application, and during deployment, some of them may be enabled while others are disabled. These models serve on the same route, although each of them has a different input and output schema. To handle this, I have built an inference graph and used a fake service as the unified entrance. Inside the entrance service I read config on startup and load dependency services on demand. However, I always got error ‘*Dependancy must be accessed as a class attribute*’ when handling incoming requests and calling dependency services, which seemed to indicate runtime service loading is prohibited. How can I accomplish it or is there any other solutions for my use case? Here is my application:

```python
@bentoml.service()
class Summarization:
    def __init__(self) -> None:
        self.pipeline = pipeline('summarization')

    @bentoml.api()
    async def summarize(self, text: str) -> str:
        result = self.pipeline(text)
        return result[0]['summary_text']

@bentoml.service()
class ImageClassification:
    def __init__(self) -> None:
        self.pipeline = pipeline('image-classification')

    @bentoml.api()
    async def classify(self, image: bytes) -> Dict[str, float]:
        return self.pipeline(image)

@bentoml.service()
class EntranceService:
    def __init__(self) -> None:
		conf = load_conf()
		if conf.enable_summarization:
		    self.summarization = bentoml.depends(Summarization)
		if conf.enable_image_classification:
		    self.image_classification = bentoml.depends(ImageClassification)

    @bentoml.api(route='/predict')
    async def run(self, input: Dict[str, Any], ctx: bentoml.Context):
        model_id = ctx.request.headers.get('model-id')
        if model_id == 'summarization' and self.summarization is not None:
			# ERROR HERE !!!
			# RuntimeError: Dependancy must be accessed as a class attribute
            return await self.summarization.summarize(input['text'])
	    elif model_id == 'image_classification' and self.image_classification is not None:
            return await self.image_classification.classify(input['image'])
        else:
            raise RuntimeError(f'unknown model id: {model_id}')
```

Hello, BentoML currently does not support dynamic startup of a service. In BentoCloud, we have a feature that support scaling to zero, meaning a service will have 0 instance when the load is low. Could you share your use case of why you would like to have dynamic service creation?

I have many models in this project, which cannot be loaded all due to the limitation of GPU number and memory. Consequently, I enable some of them in one machine and others in another machine, which is determined by a config that the application reads at startup. To this end, I have to load services after reading config.

Which means you need to swap models in and out, is that right?

Not exactly, that’s the ideal solution and is far better than my expectation (It would be a huge surprise if this feature is supported)! 😃 Currently what I’m doing is asymmetric deployment: I have all models/code in one project (Say model A, B, C, D, E), but only A + B + C are deployed in machine 1, and model C + D + E are deployed in machine 2 (Say I have two machines, none of which can accommodate all the models. Plus, model C is more popular so it’s deployed on both). And once models are deployed, they won’t change unless the application is restarted. (i.e, model swapping in/out is not needed)