This message was deleted.
# eventss
Slackbot
08/23/2022, 6:28 AMThis message was deleted.
s
Sean
08/23/2022, 6:31 AMHi Sam, currently models must be loaded back to memory to use the
save_models
Sam Joel
08/23/2022, 6:35 AMHi Sean, the problem here is we are having the inference script (script which would run the inference on the model) and the model_file separately and we would like to load the model when we start the server. If I can save the model directly then I could access the model in the serving script and then load it using the model_load_function from the inference script
Sam Joel
08/23/2022, 6:35 AMThe inference script we are having has four functions preprocess, model_load, predict and postprocess.
Sam Joel
08/23/2022, 6:38 AMIf we have to load the model and then save it I would have to extract the model_load function from the inference script and then save the model, which is a bit complex so looking if there is an alternative.
Our current method :
model = inference_script.model_load_function(.pth_file)
bentoml.pytorch.save_model(model)
s
Sean
08/23/2022, 6:44 AMI see. Ideally
save_model.pth.pthSean
08/23/2022, 6:44 AMBut I understand due to restrictions, this may not be possible.
s
Sam Joel
08/23/2022, 6:46 AMYeah exactly. We are getting the pretrained models in their respective library save formats
Sam Joel
08/23/2022, 6:46 AMSo there is a need to recreate the model object from the inference script users are providing
s
Sean
08/23/2022, 6:48 AMLoading model back into memory is usually fairly standard. Using PyTorch, it should be a few lines of Python code, if you do not wish to invoke the inference script.
s
Sam Joel
08/23/2022, 6:53 AMYeah, just looking if some simpler alternatives exist. Thanks @Sean.
👍 1