This message was deleted.

Could you please share the exact error message?

Here is the traceback with the error message. It's important to mention that this package is mentioned in my PYTHONPATH and also tried with the sys.path Also important to mention again that it works in all ways except for

bentoml serve

It seems you are using pickle models or pytorch models right?

pytorch

even though you can run the runner directly, the saved model is not a valid one. This is because of the limitation of pytorch.

What would you suggest as a solution? It worked fine with v0.13, it is working when I run it in any way other than bentoml serve

pytorch is using pickle to save/restore the model object. It works only if the definetion of the model class is in the current python module

My model requires a certain package which is available in my environment

Can you show me an example on how to use external_modules?

I think this is what Jiang meant. The module of your model should be passed as an external module during

save_model

. Otherwise, saved model files are incomplete and

load_model

will throw model not found errors.

def save_model(
    name: str,
    model: "torch.nn.Module",
    *,
    signatures: ModelSignaturesType | None = None,
    labels: t.Dict[str, str] | None = None,
    custom_objects: t.Dict[str, t.Any] | None = None,
    external_modules: t.List[ModuleType] | None = None,
    metadata: t.Dict[str, t.Any] | None = None,
) -> bentoml.Model:
    """
    Save a model instance to BentoML modelstore.

    Args:
        name (:code:`str`):
            Name for given model instance. This should pass Python identifier check.
        model (:code:`torch.nn.Module`):
            Instance of model to be saved
        signatures (:code:`ModelSignaturesType`, `optional`, default to :code:`None`):
            A dictionary of method names and their corresponding signatures.
        labels (:code:`Dict[str, str]`, `optional`, default to :code:`None`):
            user-defined labels for managing models, e.g. team=nlp, stage=dev
        custom_objects (:code:`Dict[str, Any]]`, `optional`, default to :code:`None`):
            user-defined additional python objects to be saved alongside the model,
            e.g. a tokenizer instance, preprocessor function, model configuration json
        external_modules (:code:`List[ModuleType]`, `optional`, default to :code:`None`):
            user-defined additional python modules to be saved alongside the model or custom objects,
            e.g. a tokenizer module, preprocessor module, model configuration module
        metadata (:code:`Dict[str, Any]`, `optional`,  default to :code:`None`):
            Custom metadata for given model.

Yea I understand, I am asking on how to use external_modules. How should I define it?

I'm here now

I will try it immediately and let you know 🙂

Yeah. It's the limitation of pytorch that the model has to be shipped with the source code. There's another approach to solve this. https://stackoverflow.com/questions/59287728/saving-pytorch-model-with-no-access-to-model-class-code

Using the external_modules solved the problem 🙂 Thank you very much for the quick help