This message was deleted.

cc @larme (shenyang)

can you make the tokenizer data as a partial kwargs?

Hi Suhas, could you upgrade BentoML to the latest version by

pip install -U bentoml

? The latest BentoML shouldn't have this limitation

I have version 1.0.13, still i get these issue

Hi Suhas, that part of codes should only be invoked when the model input type description is a tensor type. Maybe there's a hidden bug. We will appreciate if you could provide a minimal example to reproduce this behaviour. Thanks!

For example: you can convert bert-base model to onnx, create onnx runner with providers CUDA load tokenizer as below

from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokeinzer_data=tokenizer.tokenize("I have a new GPU!")

Initialise model locally For inference use onnx_runner onnx_runner.run.run(tokeinzer_data)

Could you try that?

tokens = list(tokenizer("this is a sample", return_tensors="np").values())
runner.run.run(*tokens)

Thanks it work's

Thanks for the feedback. We will add a bert/transfomers ONNX example in our docs.

Perfect, providing example would help new developers. Thanks for looking into