This message was deleted.

Hi @Ashish Singh, if you are deploying into a Kubernetes cluster, you can look into BentoML Yatai for solutions on resource management and isolation. Yatai provides CRDs that allows you to build bentos into OCI images and deploy as Service on Kubernetes. The CRDs also allows you to manage resource requests and limits of your deployed service.

Hey @Sean Thanks for your input. As my company uses Azure stack, which means Azure Blob Storage I can't use Yetai off the bat, but I am looking at integrating bentoml and using only yetai deployment to deploy service and runners into kubernetes. 🙏

Using only

yatai-deployment

is definitely viable. You’ll just have to pre-build and push images to a container registry, e.g. ACR, beforehand. Please keep us in the loop with your findings.

That is the work around I found and tried to deploy, it worked but we have an automated deployment process where we generate the helm file, which can be tricky with bento svc and runner definition. Also is there a way we can use yetai client class or api that can help us to generate the

BentoDeployment

if we specify the docker container image and authorization + list of runner names and runnable type?

Glad to hear to you got BentoDeployment to work. There isn’t such utility for generating the YAML

BentoDeployment

CR. We may work on similar capabilities for Kubeflow 1.8 release. If you are looking to automate the process now, I would start with the

bento.yaml

defined in the bento. It contains all the information needed for generating the svc and runner definition.

Thanks that helpful, I'll try to look into

bento.yaml

if it has essential runner mapping with a service, I plug it into a template and make it work 🙏 . Thanks for all the help