Hello everyone, I was trying to use the `on_deploy...
# ask-for-help
c
Hello everyone, I was trying to use the
on_deployment
hook to initialize some common objects to be shared by many workers. When I do that, the objects created inside the function are just local so I thought about storing them in a global var (at the top level of the file). Is that the right way to do it, or is there any way to store them as class objects? I tried this last thing, as if it would be a static method, like the code below, but when running it fails
download_model
because it says
AttributeError: 'Service' object has no attribute 'ARTIFACTS'
BentoML version 1.3.0
Copy code
@service
class MyService:

    ARTIFACTS = None  # Shared by all instances

    @on_deployment
    def download_model() -> None:
        MyService.ARTIFACTS = {
            "model": AutoModel.from_pretrained(model_name).to(device("cpu")),
            "tokenizer": AutoTokenizer.from_pretrained(model_name),
        )
        MyService.ARTIFACTS["model"].eval()

    @api
    def compute(self, texts: list[str]) -> list[list[float]]:
        ...
I have tried several other ways, and it seems the only way is having a global variable, but I am trying to avoid that as much as possible.
Finally I downloaded the model locally in the on_deployment and then read it in the
___init___
method, which is not that fast.