Hello BentoML community, I am looking for advice and help to understand your service I am part of an organization that is looking to simplify and scale our deployments. We are currently deploying with VertexAI endpoints on GCP but we'll soon be multicloud so I'm looking for other options. I have a few high level questions: • Do you know if BentoML can run on VertexAI perhaps as a custom container? It looks doable but I wasn't able to find any examples of this • I know it's possible to deploy via Kubernetes. Can anyone speak to the complexity of running BentoML on Kubernetes and how much Kubernetes knowledge is required to do so? • If I'm looking for an easier deployment option such as a Docker deployment without Kubernetes, what are my options to make this available if my company is using GCP? • From what I gather, BentoCloud is a managed service that deploys models to Kubernetes + access to a nice UI. Is this roughly a good picture? Any help would be welcome, thanks a lot!

Hi Julian, let’s chat more! will DM to follow up! Quick answers: 1. It is possible to run BentoML container on VertexAI. All the API and model inference/serving implementation should work, but things related to GPU resourcing, external queues, async tasks, concurrency control, autoscaling won’t work with VertexAI out of the box and requires some custom work to implement 2. We are still working on Yatai 2.0 for our OSS K8s support https://github.com/bentoml/Yatai/issues/504 currently we don’t recommend Yatai 1.x unless you’re using an older version of BentoML. It is possible to roll your own Kubernetes deployment solution with BentoML, but beware the same limitation as VertexAI solution listed above. 3. BentoCloud supports GCP, and supports deploying to customer’s own GCP account byoc to BentoCloud: Privacy, Flexibility, and Cost Efficiency in One Package 4. Yes! Plus a lot more, such as modular scaling of multi-service deployments, or fast scaling infrastructure and cold start optimization: https://www.bentoml.com/blog/scaling-ai-model-deployment

Thanks a lot for the detailed answers! This helps a lot