https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
j
not sure if this will work, but try
Copy code
PYTHONPATH=. python project/wf/services/save_model.py
šŸ™ 1
s
ohhhh - this seems to be working, but I get another error:
Copy code
{
  "asctime": "2023-04-11 17:47:53,414",
  "name": "flytekit",
  "levelname": "WARNING",
  "message": "FlyteSchema is deprecated, use Structured Dataset instead."
}
This is just a warning though - not sure why itā€™s throwing an error?
@Jim Rohrer
j
Not sure on that one...are you specifically using Flyte anywhere or is it some subdependency?
s
We use flyte workflow to train the model:
Copy code
import torch
import os
import typing
from flytekit import workflow
from project.wf.main import Hyperparameters
from project.wf.main import run_wf

_wf_outputs=typing.NamedTuple("WfOutputs",run_wf_0=torch.nn.modules.module.Module)
@workflow
def wf_40(_wf_args:Hyperparameters)->_wf_outputs:
	run_wf_o0_=run_wf(hp=_wf_args)
	return _wf_outputs(run_wf_o0_)
But thatā€™s just a warning - not sure why it would throw an error? Strange - Iā€™ve never run into this issue before
save_model.py
is just taking the already trained model thatā€™s saved locally:
Copy code
import torch
import bentoml

with open("/userRepoData/taeefnajib/PyTorch-MNIST/sidetrek/models/7b12772142b14606286a26751d27b878.pt", "rb") as f:
    model = torch.load(f)
    saved_model = bentoml.pytorch.save_model("example_model", model)
Maybe Iā€™m not really understanding how bentoml saves the model? Wouldnā€™t the above file just take the trained model and save it locally in 
/bentoml
? Why does bentoml care what other files exist in this case?
Or since project files here are irrelevant (i.e. the model is already trained and saved in a local file), maybe I need to exclude all other project files in bentofile.yaml ?
j
could be that flyte is getting included with the pytorch PT file?
s
Hmm thatā€™s possible - but I was able to deploy bentoml before using flyte trained models before
Strange this is suddenly happening
And I think itā€™s unlikely the pt file would include flyte
j
yeah that's really weird. unfortunately i'm not real familiar with using Flyte
same, I can't see why it would get packaged up in there
s
Right - a quick question. Is bentofile.yaml used during save_model?
It shouldnā€™t no? It seems to be a build configuration?
j
correct, I don't think so. is flyte one of your dependencies in your bentofile?
s
No but itā€™s also failing during save_model, not during build
Iā€™m having hard time understanding why even the original error would occur during save_model since Iā€™m only supplying the function with built 
.pt
file. Why would bentoml try to read the original project files?
j
just looking at an sklearn bento model I created, it just copied the .pkl file over and created a model.yaml file. If you look in 
~/bentoml/models
you should see a folder for your model there
if you look in that folder, you'll see all the versions its created....go into the latest version and you'll see the files it saved for the model, and a model.yaml file
can you copy the model.yaml file here?
s
Copy code
name: example_model
version: 7rsqyxgysc6hk3uw
module: bentoml.pytorch
labels: {}
options:
  partial_kwargs: {}
metadata: {}
context:
  framework_name: torch
  framework_versions:
    torch: 2.0.0
  bentoml_version: 1.0.15
  python_version: 3.10.10
signatures:
  __call__:
    batchable: false
api_version: v1
creation_time: '2023-04-11T17:47:53.973669+00:00'
Ohh wait - maybe itā€™s not really a bentoml error
Iā€™m running this as a child process in node server as part of the automation
Maybe it just prints out the warning but itā€™s treated as stderr, which is considered an error in our server
Sorry I think itā€™s my mistake šŸ™‡ā€ā™‚ļø
j
No worries! Sometimes you just have to talk it out šŸ™‚
s
Thank you so much for walking me through this! Much appreciated. But just for future reference, do you know why the original error of 
module "project" not found
error happened during save_model even though itā€™s only using the saved local 
pt
file?
Does bentoml reference the original code during save_model for some reason?
For example, if I have no code - just local 
pt
file, shouldnā€™t bentoml save_model still work?
j
nah, this is just Python's crappy module structuring lol...it is weird that save_model.py doesn't seem to reference anything else in your package structure, so I'm not sure why it would matter if it can find your 
project
module
you might need an 
__init__.py
file in your 
services
directory? not 100% sure on that one
s
Ahh ok - yeah Iā€™ve been fighting the python module import system for some time now so I know what thatā€™s like lol
j
haha it's so bad
šŸ˜„ 1
s
Canā€™t believe how hard it is to just import something in python
Thanks again for your help - really appreciate this!
j
most welcome šŸ™‚
šŸ™ 1
s
@Jim Rohrer Sorry just one more quick question - does save_model have a REST API counterpart?
Also for bentoml cli commands including the build?
j
Hi @Seungchan Lee , is 
bentoml models import
what you ask?
s
@Jiang No I mean if thereā€™s a REST API endpoint I can hit to do the same thing instead of running a python script or using bentoml cli. Itā€™s just harder to automate bentoml via CI/CD when you have to run a subprocess to run them (much slower since spawning a subprocess takes 2-4 seconds to start and also harder to deal with responses, errors, etc). For example, Iā€™m running a python script to use 
save_model
as part of a CI/CD automation, but in order to get the resulting bento model version generated by 
save_model
, I have to 
print()
to stdout and parse that which is not great. Also a simple warning prints to 
stderr
which makes things more brittle as itā€™ll be treated as an error and fail to proceed to next step.
šŸ± 1
Please let me know if thereā€™s a REST api for bentoml so I can replace the scripts and cli subprocesses
j
Unless youā€™re running Yatai in Kubernetes, thereā€™s no REST API equivalent for the bento cli commands, at least none that Iā€™m aware of.
Theoretically you could build a REST wrapper to execute cli commands, but youā€™d ultimately be doing the same thing, calling sub processes.
šŸ‘ 1
a
Can you try adding importlib.import_module(ā€œprojectā€) to external_modules
j
@Seungchan Lee Hi Seungchan. 
save_model
is actually part of the SDK, not the command line. The difference here is that save_model is used to save a model object located in Python memory to a file, and therefore it must be executed with code at the end of the training pipeline. If we want to support the REST API as you mentioned, we first need a common convention for sending Python objects in memory via the REST API. However, as we all know, such a convention does not exist. If you have a private protocol, you can implement this REST server yourself, but it cannot be promoted to the community for everyone to use. I'm not sure if I fully understand your question. Can you provide a more specific scenario to help me understand better?
s
@Aaron Pham Jim Rohrer actually helped me with the original problem. Thanks for suggesting a solution though!
šŸ» 1
@Jiang Right that makes sense - I didnā€™t think through the 
save_model
part. As for the motivation for wanting REST api for cli commands is for easier automation.
Running bentoml cli requires spawning subprocesses, which is harder to work with than a REST api. Itā€™s slower and harder to work with responses (since stdout is just text that needs to be parsed) as well as handling errors (stderr is really tricky to handle)
Also harder to grab model/bento data from our internal dashboard
j
Sure, that will be great. Kudos to @Aaron Pham
Open in Slack
PreviousNext
Powered by