C S
08/16/2024, 5:09 PMon_deployment hook to initialize some common objects to be shared by many workers. When I do that, the objects created inside the function are just local so I thought about storing them in a global var (at the top level of the file). Is that the right way to do it, or is there any way to store them as class objects?
I tried this last thing, as if it would be a static method, like the code below, but when running it fails download_model because it says AttributeError: 'Service' object has no attribute 'ARTIFACTS'
BentoML version 1.3.0
@service
class MyService:
ARTIFACTS = None # Shared by all instances
@on_deployment
def download_model() -> None:
MyService.ARTIFACTS = {
"model": AutoModel.from_pretrained(model_name).to(device("cpu")),
"tokenizer": AutoTokenizer.from_pretrained(model_name),
)
MyService.ARTIFACTS["model"].eval()
@api
def compute(self, texts: list[str]) -> list[list[float]]:
...C S
08/16/2024, 7:07 PMC S
08/16/2024, 9:50 PM___init___ method, which is not that fast.