Hi all. I have a general question about the intern...
# ask-for-help
k
Hi all. I have a general question about the internals of yatai/bentoml. Can I use the same runner with different Bentos and subsequent deployments, if yes how can I access it and If I adjust the runner configs from one deployment does it affect the other deployments as well (e.g. min/max replica)? NOTE: I am not talking about models, I am talking about runners. My use case is the following, I want to be able to modularize each model in its own endpoint, but also in rare cases, I want to combine all my models (with an inference graph, some of them sequentially and some of them in parallel) to create the end result. Since I have several models with multiple requirements I do not want to combine all of them in a single bento. It is my understanding that you can choose whichever model you want and convert it to a runner but you cannot get the deployed runner in some way. As an extension of this question, can I use the endpoint of another deployment inside a bento/deployment? I guess this can be done by using requests (and the appropriate endpoint) in the service endpoint, right? So my general question is how can you combine runners to be accessed from different deployments, in order to remove duplicate runners and logic? Or is bento supposed to be completely individual and not be combined in any way? Thanks!