Hello, I've been using the open-source version of ...
# ask-for-help
a
Hello, I've been using the open-source version of BentoML for a while. Recently, we encountered errors with the
bentoml containerize
command. The issue stemmed from using remote buildx without a local installation—buildx was performing a health check by testing local installation of docker. This raises a question: What is the goal of
bentoml build
and
containerize
, and why not simply use a standard Dockerfile? In particular, I am surprised by the new
PythonImage
that enable to define the Dockerfile in python. Sometimes it seems like we end up with a standard Docker image that only contains the models and the corresponding files. What does it solve compared to creating a docker image from a Dockerfile ? I feel like I am missing some of the point. Some of my guesses: Data scientists don't know how to create dockerimage. It helps handling some of the kube orchestration. It helps define a business logic that calls the various services and have it work remotely and locally. Thank you for BentoML 🙂
👍 1
c
Hi @Arnault Chazareix - BentoML does generate a standard Dockerfile in the bento folder, which can be used directly for scenarios like this where users need more flexibility
Give this a try:
Copy code
cd $(bentoml get summarization:latest -o path)
docker build -t mybento -f env/docker/Dockerfile .
We designed this for easier devX and for providing a standardized abstraction for other potential infrastructure optimization. BentoCloud for example leverages this abstraction to use our proprietary image format and container runtime to speed up cold start times. Another good example is Nvidia’s Dynamo: https://github.com/ai-dynamo/dynamo/tree/main/deploy/dynamo/sdk
👍 1
And yes, for multi-model, multi-component services, with a mix of model inference and business logic, this significantly simplifies the deployment process
👍 1
a
I resulted in using docker directly to solve the issue. That's what arose the question. 🙂
thanks for the pointers. I will look into these
Is there a simple way to get a helm chart out of a distributed service ?
c
not yet, currently needs a bit manual work but we are working on something like that for OSS users
🍱 1
👍 1