https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
gratitude thank you 1
j
Hello! everything should still work as it is! if you are still using 1.1 so there is no need to migrate
But yes, thats a good point
m
Thanks, and is yatai compatible with 1.2?
Doesn’t look like, right?
h
Hey @Jian Shen Yap Thanks for the quick reply, we noticed some incompatibilities with the cli. E.g. 
bentoml yatai login
does not work any longer. More importantly, however, it seems that runners are deprecated as well as adaptive batching? Is this correct? But, in Yatai we still have the concept of runners, is this something that will be deprecated too?
v
➕ I'm also wondering what happens to yatai compatibility and adaptive batching in bento 1.2
m
➕
c
@Vivien Robert @Henrik Jilke both the runner architecture and adaptive batching are still natively supported! With 1.2, you can define multiple services and specify service “dependencies”, this design actually makes it a lot more flexible to use this architecture, allowing a few patterns that’s not possible before
Here’s an example using dynamic batching: https://github.com/bentoml/BentoClip/blob/main/service.py#L26
Here’s an example using multiple services (runner equivalent): https://github.com/bentoml/BentoSD2Upscaler
h
@Chaoyu Thanks for the reply. Lets say
Copy code
@bentoml.service(name="iris_classifier"})
 class IrisClassifier:
     iris_model = bentoml.sklearn.get("iris_sklearn:latest").load_model()

     @bentoml.api
     def classify(self, input_series: np.ndarray) -> np.ndarray:
         return this.iris_model.predict(input_series)
In the 
BentoDeployment
CRD how would we define runners, e..g what to put here?
Copy code
runners:
        - name: "${VAL}"
And lets say we want to have embedded runners, how would we specify this?
m
Copy code
$ kubectl explain bentodeployment.spec.runners
That gives you a pretty good idea of what can go into the runners section.
Thanks for the quick reply, we noticed some incompatibilities with the cli. E.g. 
bentoml yatai login
does not work any longer.
@Henrik Jilke you've probably figured it out by now, but I just discovered that 
bentoml yatai login
is now simply 
bentoml cloud login
. It still work the same as before.
v
Well yes bentoml cloud login works but none of the other remote related functionalities worked with a 1.2 bento client version and a yatai installation (once logged in, I could not manage to pull/push models or list deployments)
m
I concur. 😢 Weirdly, I was able to push a newly built bento up to yatai, but I'm not able to pull it back down. And I haven't tried deploying it, but have been told that it likely won't succeed.
For now we are forced to remain on v1.1 in order to continue to use yatai. @Chaoyu has assured me that critical bug fixes will continue to be made for the 1.1 branch, so that's something. But of course it would be great to be able to start using the new 1.2 functionality with yatai. I've offered my own help with re-establishing compatibility, but it's likely not something I would be able to do without assistance from BentoML.
Open in Slack
PreviousNext
Powered by