This message was deleted.
# ask-for-helps
Slackbot
07/31/2023, 10:02 PMThis message was deleted.
c
Chaoyu
08/01/2023, 2:22 AMyou can serve it with BentoML by implementing your own runnable class: https://docs.bentoml.com/en/latest/concepts/runner.html#custom-runner
Chaoyu
08/01/2023, 2:22 AMsaving the model with BentoML model store is slightly more complex, we don’t currently have detailed documentation
Chaoyu
08/01/2023, 2:23 AMbut it’s possible for users to do this - you may take a look at the scikit-learn module’s implementation, which can be a good starting point
k
Kshitij Chawla
08/01/2023, 3:15 PMThanks. Do we have to use the model store or can we plugin our own method/library?
c
Chaoyu
08/01/2023, 4:33 PMyou don’t have to, it’s nice for version management if you iterate with retraining/fine-tuning a lot
Chaoyu
08/01/2023, 4:34 PMbut if you are using pre-trained model from NNUnet, it’s probably ok to just package the files in the Bento, load with custom runner, without the modelstore
k
Kshitij Chawla
08/01/2023, 4:36 PMAlso, is there no way to just track general models (such as for my use case) without associating it with learning framework? For example instead of
bentoml.pytorch.save()bentoml.generic.save()a
Aaron Pham
08/02/2023, 8:13 PMyou can use
bentoml.models.create👌 1
👍 1
Aaron Pham
08/02/2023, 8:13 PMyou will just need to manage the metadata and context as well as framework version yourself
Aaron Pham
08/02/2023, 8:13 PMto get the model back you can do
bentoml.models.getk
Kshitij Chawla
08/02/2023, 8:19 PMThanks so much!
a
Aaron Pham
08/02/2023, 8:47 PMA very extreme example (idk if this is useful for you) of this in action:
https://github.com/bentoml/OpenLLM/blob/72337410cfebcd5f348dd4d5d1ac62e93966bf08/src/openllm/serialisation/transformers.py#L96