This message was deleted.

I know you can load a MLflow model from a

run_id

(as shown in this notebook) but as I’m working with NLP models a lot, I tend to pack everything up into a pyfunc (model + tokenizer) and can simply load the model using

mlflow.pyfunc.load_model

and its

.predict()

method. It would be ideal to be able to wrap these pyfunc models around a BentoML service and use it for deployment. Technically I can create a custom MLflow artifact and service extracting the

.predict()

from the pyfunc but this is not ideal cause pyfunc forces you to define your predict that way

class MyModel(mlflow.pyfunc.PythonModel):
    def load_context(self, context):
        # load your artifacts
    def predict(self, context, model_input):
        return my_predict(model_input.values)

and that

context

parameter is messing up with the

predict()

method I need for BentoML

PyFunc support is already supported in the 1.0, we are still working on related documentation tho

thanks! I found it and started playing around with the beta version I found an issue with the loading method that I just filed here in github

You can import the entire PyFunc model as a BentoML model:

bentoml.mlflow.import_from_uri(...)

And then load the PyFunc model back as a Runner and use it in your Service definition:

runner = bentoml.mlflow.load_runner("name:version")

my_svc = bentoml.Service('my_service', runners=[runner])

@my_svc.api(...)
def predict(...):
    runner.run( .. )