Private github/bitbucket repos seems broken in bento v2. When forcing bento v1 during

bentoml build

by adding an empty Dockerfile.template in the build.docker section, bentoml successfully builds my private repo as source distribution and I can containerize my image, however, removing that empty Dockerfile.template and thereby using bento v2, it does not prebuild my private repo, and when it install dependencies when containerizing, it is unable to install it as it does not mount my SSH when running this command:

RUN --mount=type=cache,target=/root/.cache/ uv pip install -r ./env/python/requirements.txt

. What can I do to make it work with v2?

[project]
...
dependencies = [
    "ai-utils @ <git+ssh://git@bitbucket.org/unumed/ai_utils.git@v0.5.1>",
    "bentoml[tracing-otlp]==1.3.22",
...

[tool.bentoml.build.docker]
system_packages = ["git", "openssh-client"]
dockerfile_template = "Dockerfile.template"  # Empty dockerfile

Here's the output from

bentoml build

forcing the v1:

(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly-detection$ uv run bentoml build
WARNING: docker.dockerfile_template is not supported by bento v2, fallback to bento v1
INFO: Adding current BentoML version to requirements.txt: bentoml==1.3.22
INFO: Locking PyPI package versions.
Building source distribution...
Successfully built dist/ai_utils-0.5.0.tar.gz
INFO: Built sdist ai_utils-0.5.0.tar.gz
...
Containerizes without issues

And here's with v2:

(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly-detection$ uv run bentoml build
INFO: Locking PyPI package versions.
...
(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly-detection$ uv run bentoml containerize anomaly-detection --opt ssh=default --verbose
...
#14 [base-container 10/14] RUN --mount=type=cache,target=/root/.cache/ uv pip install -r ./env/python/requirements.txt
#14 0.934 Using Python 3.10.16 environment at: /app/.venv
#14 0.991    Updating <ssh://git@bitbucket.org/unumed/ai_utils.git> (68123380fb9465ec816a1ba9f66af0aea6126603)
#14 1.463   × Failed to download and build `ai-utils @
#14 1.463   │ <git+ssh://git@bitbucket.org/unumed/ai_utils.git@68123380fb9465ec816a1ba9f66af0aea6126603>`
#14 1.464   ├─▶ Git operation failed
#14 1.464   ├─▶ failed to fetch into: /root/.cache/uv/git-v0/db/6027dcdb77c1efab
#14 1.464   ├─▶ failed to fetch branch, tag, or commit
#14 1.464   │   `68123380fb9465ec816a1ba9f66af0aea6126603`
#14 1.465   ╰─▶ process didn't exit successfully: `/usr/bin/git fetch --tags --force
#14 1.465       --update-head-ok '<ssh://git@bitbucket.org/unumed/ai_utils.git>'
#14 1.465       '+refs/heads/*:refs/remotes/origin/*' '+HEAD:refs/remotes/origin/HEAD'`
#14 1.465       (exit status: 128)
#14 1.465       --- stderr
#14 1.465       Host key verification failed.
#14 1.465       fatal: Could not read from remote repository.
#14 1.465 
#14 1.465       Please make sure you have the correct access rights
#14 1.465       and the repository exists.
#14 1.465 
#14 ERROR: process "/bin/sh -c uv pip install -r ./env/python/requirements.txt" did not complete successfully: exit code: 1
------
 > [base-container 10/14] RUN --mount=type=cache,target=/root/.cache/ uv pip install -r ./env/python/requirements.txt:
1.465       --update-head-ok '<ssh://git@bitbucket.org/unumed/ai_utils.git>'
1.465       '+refs/heads/*:refs/remotes/origin/*' '+HEAD:refs/remotes/origin/HEAD'`
1.465       (exit status: 128)
1.465       --- stderr
1.465       Host key verification failed.
1.465       fatal: Could not read from remote repository.
1.465 
1.465       Please make sure you have the correct access rights
1.465       and the repository exists.
1.465 
------
Dockerfile:39
--------------------
  37 |     COPY --chown=bentoml:bentoml ./env/python ./env/python/
  38 |     # install python packages
  39 | >>> RUN --mount=type=cache,target=/root/.cache/ uv pip install -r ./env/python/requirements.txt
  40 |     
  41 |     RUN chmod +x ./env/docker/script__b7277a60a376858da8de2cb2eded9439 && ./env/docker/script__b7277a60a376858da8de2cb2eded9439
--------------------

Hi, I'm also running into this issue. As a workaround in the newly released 1.4, you can remove the private repo(s) from dependencies and install it separately by running a custom command:

my_image = (
    bentoml.images.PythonImage(python_version="3.11")
    .system_packages("openssh-client")
    .run("mkdir -p -m 0700 ~/.ssh && ssh-keyscan <http://github.com|github.com> >> ~/.ssh/known_hosts")
    .python_packages("torch", "transformers")
    .run(
        "--mount=type=cache,target=/root/.cache/ --mount=type=ssh uv pip install "<git+ssh://git@github.com/username/repository.git@branch_name>""
    )
)

@bentoml.service(image=my_image)
class MyService:

Haha awesome, what a workaround 😄 My problem is that I use the repo for other stuff as well - I could put it in an optional dependency group and do your workaround. Thank you for the suggestion!