Hi I m trying to save a transformer model and tokenizer But

Question

Hi I m trying to save a transformer model and tokenizer But I am getting error I wanted to use T5 but its similar for all I cannot save tokenizer ```import bentoml from transformers import AutoModel AutoTokenizer sum model = AutoModel from pretrained bert base uncased tokenizer = AutoTokenizer from pretrained bert base uncased bento model = bentoml transformers save model email sum model sum model bento tokenizer = bentoml transformers save model email sum tokenizer tokenizer ``` ```Traceback most recent call last File mnt ssd mamba envs sum bin bentoml line 8 in lt module gt sys exit cli File mnt ssd mamba envs sum lib python3 10 site packages click core py line 1130 in call return self main args kwargs File mnt ssd mamba envs sum lib python3 10 site packages click core py line 1055 in main rv = self invoke ctx File mnt ssd mamba envs sum lib python3 10 site packages click core py line 1657 in invoke return process result sub ctx command invoke sub ctx File mnt ssd mamba envs sum lib python3 10 site packages click core py line 1404 in invoke return ctx invoke self callback ctx params File mnt ssd mamba envs sum lib python3 10 site packages click core py line 760 in invoke return callback args kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml cli utils py line 334 in wrapper return func args kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml cli utils py line 305 in wrapper return value = func args kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml cli utils py line 262 in wrapper return func args kwargs File mnt ssd mamba envs sum lib python3 10 site packages simple di init py line 139 in return func inject args bind args inject kwargs bind kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml cli bentos py line 323 in build bento = Bento create File mnt ssd mamba envs sum lib python3 10 site packages simple di init py line 139 in return func inject args bind args inject kwargs bind kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml internal bento bento py line 182 in create svc = import service File mnt ssd mamba envs sum lib python3 10 site packages simple di init py line 139 in return func inject args bind args inject kwargs bind kwargs File mnt ssd mamba envs sum lib python3 10 site packages bentoml internal service loader py line 137 in import service module = importlib import module module name package=working dir File mnt ssd mamba envs sum lib python3 10 importlib init py line 126 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1050 in gcd import File lt frozen importlib bootstrap gt line 1027 in find and load File lt frozen importlib bootstrap gt line 1006 in find and load unlocked File lt frozen importlib bootstrap gt line 688 in load unlocked File lt frozen importlib bootstrap external gt line 883 in exec module File lt frozen importlib bootstrap gt line 241 in call with frames removed File mnt ssd projects AIESA bento deploy service py line 30 in lt module gt bento tokenizer = bentoml transformers save model File mnt ssd mamba envs sum lib python3 10 site packages bentoml internal frameworks transformers py line 867 in save model framework pretrained framework AttributeError BertTokenizerFast object has no attribute framework ``` I m pretty new to bentoml Am I doing anything wrong or what is the cause Thanks in advance

Aaron Pham · Answer

oh sorry this is def a bug Let me put up a PR for this Thanks for spotting this

Aaron Pham · Answer

For now what you can do is to save the tokenizer as a custom object ```bentoml transformers save model model model custom objects= tokenizer tokenizer ``` And you can access this tokenizer like so ```bentomodel = bentoml models get model tokenizer = bentomodel custom objects tokenizer ```

Nuwan · Answer

Thank you I will it +1

Aaron Pham · Answer

I notice that you are also running save inside service py This is usually discouraged because `service py` will be forked to child process when running `bentoml serve` and you probably don t want to save the model everytime you serve the service

Aaron Pham · Answer

Usually save should be done after training but for the case of transformers you can create a separate file to run before the actual serving time

Nuwan · Answer

Oh Thank you for information I will use separate file for it