https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
🍱 1
👀 1
a
Hi @Shihgian Lee, BuildKit is the more efficient caching layer from the moby project. It helps with support for external caching as well as a new build graph solver. What this means it it improve build time significantly by running in parallel. I’m wondering what is the CI/CD infras your team is using?
👍 1
s
Hi @Aaron Pham We are using gitlab as our CI/CD. Our DevOps has a standard build script to containerize the docker file. Does this mean we need to check the version of our docker to see if it supports it?
a
Is it possible to see what is the version of docker the CI is using? If Docker is >19.03, by using
Copy code
DOCKER_BUILDKIT: 1
Docker should be able to use buildkit
We did mention about using buildkit here, but we didn’t inside the migration guide.
👍 1
I will make a PR to note this inside our migration guide. Thanks for the feedback
👍 1
s
@Aaron Pham Thank you for the explanation. That was helpful to our DevOps. We are going to try your suggestion by adding the 
DOCKER_BUILDKIT=1 docker build .
as stated in the docker doc.
a
Please let me know if supporting buildkit on CI/CD is too big of a hassle. I’m also thinking about having options for users to generate dockerfile that doesn’t require buildkit.
❤️ 1
s
@Aaron Pham The 
DOCKER_BUILDKIT=1
flag seem to resolve the issue. But, I am getting another containerize issue. The following is my step for building the dockerfile: 1. bentoml export my_classifier:latest saved/bento.tar.gz 2. tar -xf saved/bento.tar.gz -C saved/ 3. container -d saved/env/docker # container is from our DevOps script. I am getting the following error:
Copy code
bash: /home/bentoml/bento/env/python/install.sh: No such file or directory
The actual location of the script is 
saved/env/python/install.sh
. How do we fix that?
a
what is this 
container
executable doing?
s
it is a wrapper to docker command with additional metadata we pass in but i omitted it here for brevity.
a
seems to me that it modify the home directory, you might have to change 
BENTOML_HOME
environment inside the container
Copy code
--build-arg BENTOML_HOME=saved/
you can actually pass through all docker build commands to 
bentoml containerize
as it accepts all of docker options
Copy code
docker build --build-args ... --env 

bentoml containerize bento:tag --build-args ... --env
s
we don’t use bentoml containerize. we have our own containerize which is 
container
that you referred to above.
a
sg. If 
container
accepts all docker arguments, then 
--build-arg
solution could fix the issue.
s
the reason we love bentoml 0.13 is because it was so simple and just work without making so much assumption. with 1.0, i see more assumptions being pushed to us. this makes integration with our infrastructure hard.,
how is it different from bentoml 0.13? can we make it as simple as 0.13? it just works.
sg. If 
container
accepts all docker arguments, then 
--env
solution could fix the issue.
i will have to look at the script. bentoml 1.0 is making deployment to our infrastructure harder and harder. Cc @Bo @Tim Liu @Chaoyu
t
thanks for the feedback @Shihgian Lee! This is a good issue for the team to take a look at in contrast with our 0.13 version. Definitely a couple points for sure to add to the migration guide. Will synch with the team at our next product meeting
👍 1
c
@Shihgian Lee could you share the full script you used to build the image?
looks like the build_context directory is not set to the Bento directory
s
@Chaoyu Sure. I will share the relevant steps and scripts in our gitlab-ci.yml without sharing the whole ci file.
c
In general, we recommend using 
bentoml containerize
as it is a drop-in replacement for 
docker build
and ensures all the settings are correct. It is still possible to do the manual docker build like 0.13, but it will need some extra manual settings like 
DOCKER_BUILDKIT=1
, and making sure the build_context and docekrfile path are correct
a
A minimal docker build command for your usecase
Copy code
docker build -f saved/env/docker/Dockerfile saved/
👍 2
s
@Chaoyu The following are our 
build
and 
container
stages in the gitlab ci:
Copy code
build:
  image: <http://gcr.io/python-build:3.7|gcr.io/python-build:3.7>
  stage: build
  only:
    - main
  artifacts:
    when: on_success
    expire_in: "30 min"
    paths:
      - saved/*
  script:
    - pip3 install --upgrade pip
    - pip3 install -r requirements.txt
    - gcloud auth activate-service-account --key-file=<redacted>
    - ./fetch-artifacts.sh
    - PYTHONPATH=$(pwd) python3 pack/package.py
    - bentoml build -f ./src/bentofile.yaml ./src/
    - mkdir saved
    - bentoml export demand_transformation:latest saved/bento.tar.gz

container:
  image: <http://gcr.io/java-build:8|gcr.io/java-build:8>
  stage: container
  dependencies:
    - build
  only:
    - main
  script:
    - tar -xf saved/bento.tar.gz -C saved/
    - export DOCKER_BUILDKIT=1
    - container -an my-model -v $BUILD_VERSION -dd saved/env/docker
c
What does the 
-dd saved/env/docker
parameter mean?
s
it tells the script the directory that contains the docker file.
c
You will need two parameters for the docker build command, 
-f
for the path to Dockerfile, and argument 
build_context
, which specify the directory to run the build
E.g.:
Copy code
BENTO_PATH=$(bentoml get iris_classifier:latest -o path)
docker build -f $BENTO_PATH/env/docker/Dockerfile $BENTO_PATH
s
@Chaoyu i will work with our devops tomorrow to find out how easy it is to change the script. when we do a bentoml build, can we have an argument for build context instead? then everything will be correct in the dockerfile and no burden being put on the user?
c
@Shihgian Lee this build context is something 
docker build
command requires, unfortunately there’s no way for BentoML to know that ahead of time
s
@Chaoyu The problem is in the dockerfile where we made the assumption where home directory and subsequent directory is: 
/home/bentoml/bento/env/python/install.sh
. In bentoml 0.13, I don’t believe we make this assumption right?
Specifically, the 
/home/bentoml
part is the problem.
c
Oh, that part is same as 0.13
the problem is that 
build_context
was set wrong
the only change is that 
Dockerfile
no longer lives under the root directory of Bento
My guess is that the 
container
script may also provide a separate parameter for the Dockerfile path, in addition to the 
-dd
which is for 
build context
, cause they’re very standard 
docker build
arguments
the default docker CLI behavior is to provide only the 
build_context
, and docker build command will look for 
Dockerfile
under that directory
s
@Chaoyu okay, let me sync with our devops tomorrow and see how it goes. thanks for going through it with me. we are short handed at the moment and i would like to get the first migration done asap so that we have an example.
c
let me know how it goes! sorry about all the trouble, a lot of the changes can be painful for migrating from 0.13, we had to make some of the changes for all the new features that users are asking
the buildkit part can speed up the build 6-10x on average, supports multi-arch image build, and also provide more secure ways to mount credentials during build time
s
@Chaoyu i will keep you posted. i understand. thank you!
🙏 1
A minimal docker build command for your usecase
Copy code
docker build -f saved/env/docker/Dockerfile saved/
Just saw this for our use case. Thanks @Aaron Pham!
👍 1
@Aaron Pham @Chaoyu I managed to modify the script and deployed bentoml 1.0.6 successfully to our k8s. Thanks for the help.
In general, we recommend using 
bentoml containerize
as it is a drop-in replacement for 
docker build
and ensures all the settings are correct.
As I explained previously, we have our own container script. Although I appreciate the help, BentoML should not make assumptions that we will be using the CLI tool. Also, enabling BuildKit is fine now because we are using docker. But, we are in the process of migrating to google build tool. Then BuildKit may not be available in google build. Containerization should be as generic as possible. If needed, performance optimization can be layered on top of it to target specific build tool. I am getting another error in dev environment after deployment. I will post it as a separate thread in the support.
Open in Slack
PreviousNext
Powered by