https://bentoml.com logo
Join Slack
Powered by
Torch+cpu, HOW!? I've been struggling for quite s...
# ask-for-help
t
Torch+cpu, HOW!? I've been struggling for quite some time now just trying to build my bento with pytorch on cpu (pytorch+cpu). I'm using my pyproject.toml managed by astrals uv, that looks something like this:
Copy code
[project]
name = "anomaly_detection"
version = "1.0.0"
description = "Detect all the anomalies!"
requires-python = ">=3.12, <3.13"
readme = "README.md"
dependencies = [
    "ai-utils",
    ...
    "bentoml==1.3.19",
    "torch==2.4.1+cpu",
    ...
}
...
[tool.uv.sources]
ai-utils = { git = "<ssh://git@bitbucket.org/unumed/ai_utils.git>", rev = "v0.5.1" }
torch = { index = "pytorch" }
torchvision = { index = "pytorch" }

[[tool.uv.index]]
name = "pytorch"
url = "<https://download.pytorch.org/whl/cpu>"
explicit = true
...

[tool.bentoml.build]
service = "anomaly_detection.bento.service:AnomalyDetectionService"
include = ["**/*.py"]
exclude = [
    ".devcontainer/",
    ".dvc/",
    ".git/",
    ".githooks/",
    ".history/",
...
]
...
I then run 
uv run bentoml build
getting this error:
Copy code
(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly_detection$ uv run bentoml build
   Built anomaly-detection @ file:///home/tokereines/unumed/anomaly_detection
Uninstalled 2 packages in 324ms
Installed 2 packages in 458ms
INFO: Adding current BentoML version to requirements.txt: bentoml==1.3.19
INFO: Locking PyPI package versions.
  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of torch==2.4.1+cpu and you require torch==2.4.1+cpu, we can conclude that your requirements are unsatisfiable.
Error: [bentos] `build` failed: Failed to lock PyPI packages: Command '['/home/tokereines/unumed/anomaly_detection/.venv/bin/python', '-m', 'uv', 'pip', 'compile', '/home/tokereines/bentoml/tmp/tmp_x5xqm4mbentoml_bento_anomaly_detection/env/python/requirements.txt', '--allow-unsafe', '--no-header', '--output-file=/home/tokereines/bentoml/tmp/tmp_x5xqm4mbentoml_bento_anomaly_detection/env/python/requirements.lock.txt', '--emit-index-url', '--emit-find-links', '--no-annotate', '--quiet']' returned non-zero exit status 1.
To which I've tried with
Copy code
[tool.bentoml.build.python]
extra_index_url = ["<https://download.pytorch.org/whl/cpu>"]
Which then resulted in it being unable to find the correct version of any of my other packages. I've had it work previously by pip compiling to a requirements.txt file and then installing with --extra-url and --no-deps, but that only worked if I provided another base-image of python 3.10 bookworm and bentoml version 1.3.14. I want to keep update, so I have updated bentoml to 1.3.19 as well as other packages, but now it wont build. Any idea how to fix it or in general just tips to how to make bentoml build with torch on cpu? The diff in image size is 2.6gb vs 12gb...
j
cc: @Frost Ming
f
can you paste the build error with the 
extra_index_url
applied?
t
With the extra_index_url
Copy code
(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly-detection$ uv run bentoml build
warning: `VIRTUAL_ENV=/home/tokereines/unumed/anomaly_detection/.venv` does not match the project environment path `.venv` and will be ignored
   Built anomaly-detection @ file:///home/tokereines/unumed/anomaly-detection
Uninstalled 2 packages in 149ms
Installed 2 packages in 85ms
INFO: Adding current BentoML version to requirements.txt: bentoml==1.3.19
INFO: Locking PyPI package versions.
  × Failed to build `sklearn==0.0.post12`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)

      [stderr]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.

      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error

      More information is available at
      <https://github.com/scikit-learn/sklearn-pypi-package>

      hint: This usually indicates a problem with the package or the build environment.
  help: `sklearn` is often confused for `scikit-learn` Did you mean to install `scikit-learn` instead?
Error: [bentos] `build` failed: Failed to lock PyPI packages: Command '['/home/tokereines/unumed/anomaly-detection/.venv/bin/python3', '-m', 'uv', 'pip', 'compile', '/home/tokereines/bentoml/tmp/tmpbfxxwzuwbentoml_bento_anomaly_detection/env/python/requirements.txt', '--extra-index-url', '<https://download.pytorch.org/whl/cpu>', '--allow-unsafe', '--no-header', '--output-file=/home/tokereines/bentoml/tmp/tmpbfxxwzuwbentoml_bento_anomaly_detection/env/python/requirements.lock.txt', '--emit-index-url', '--emit-find-links', '--no-annotate', '--quiet']' returned non-zero exit status 1.
Without the extra_index_url:
Copy code
(anomaly-detection) tokereines@TokeUnumed:~/unumed/anomaly-detection$ uv run bentoml build
warning: `VIRTUAL_ENV=/home/tokereines/unumed/anomaly_detection/.venv` does not match the project environment path `.venv` and will be ignored
   Built anomaly-detection @ file:///home/tokereines/unumed/anomaly-detection
Uninstalled 1 package in 0.63ms
Installed 1 package in 2ms
INFO: Adding current BentoML version to requirements.txt: bentoml==1.3.19
INFO: Locking PyPI package versions.
  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of torch==2.4.1+cpu and you require torch==2.4.1+cpu, we can conclude that your requirements are unsatisfiable.
Error: [bentos] `build` failed: Failed to lock PyPI packages: Command '['/home/tokereines/unumed/anomaly-detection/.venv/bin/python3', '-m', 'uv', 'pip', 'compile', '/home/tokereines/bentoml/tmp/tmpoywfrvfvbentoml_bento_anomaly_detection/env/python/requirements.txt', '--allow-unsafe', '--no-header', '--output-file=/home/tokereines/bentoml/tmp/tmpoywfrvfvbentoml_bento_anomaly_detection/env/python/requirements.lock.txt', '--emit-index-url', '--emit-find-links', '--no-annotate', '--quiet']' returned non-zero exit status 1.
This is my entire pyproject.toml:
Copy code
[project]
name = "anomaly_detection"
version = "1.0.0"
description = "Detect all the anomalies!"
requires-python = ">=3.12, <3.13"
readme = "README.md"
dependencies = [
    "ai-utils",
    "pandas==2.2.3",
    "pydantic>=2.9.0",
    "pyro-ppl==1.9.1",
    "bentoml==1.3.19",
    "torch==2.4.1+cpu",
    "pydantic-settings==2.2.1",
    "fastapi==0.115.6",
    "itkwidgets==0.32.6",
    "scikit-learn==1.5.2",
    "hydra-core==1.3.2",
    "seaborn==0.13.2",
    "mlflow-skinny==2.7.1", # Fixed version for build errors
    "python-dotenv==1.0.1"
]

[project.scripts]
anomaly = "anomaly_detection.cli.main:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.uv.sources]
ai-utils = { git = "<ssh://git@bitbucket.org/unumed/ai_utils.git>", rev = "v0.5.1" }
torch = { index = "pytorch" }
torchvision = { index = "pytorch" }

[[tool.uv.index]]
name = "pytorch"
url = "<https://download.pytorch.org/whl/cpu>"
explicit = true

[tool.ruff]
target-version = "py310"
line-length = 120
extend-include = ["*.ipynb"]
exclude = [".history", ".ruff_cache"]

[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401", "F403", "F405"]

[tool.ruff.lint]
extend-select = [
    "I",
    "RUF100",
    "RUF200",
    "E",
    "UP",
    "B",
    "SIM",
]

[tool.bentoml.build]
service = "anomaly_detection.bento.service:AnomalyDetectionService"
include = ["**/*.py"]
exclude = [
    ".devcontainer/",
    ".dvc/",
    ".git/",
    ".githooks/",
    ".history/",
    ".pytest_cache/",
    ".ruff_cache/",
    ".venv/",
    ".vscode/",
    "__pycache__/",
    "**/*.parquet",
    "**/*.csv",
    "dist/",
    "tests/",
    "scripts/",
    "outputs/",
    "history/",
]

[tool.bentoml.build.labels]
owner = "unumed"
stage = "demo"
I tried removing my own private 
ai_utils
repo, and then it actually builds without problems.
But why does it work with 
uv sync
locally, but not with 
uv pip
as used in bentoml?
f
Basically, all the configuration under 
[tool.uv.*]
will not be read by bentoml build. You must pretend they don’t exist when running bentoml build. Therefore, the issue is clear: first, you need 
python.extra_index_url
to install 
torch+cpu
, secondly, the pypi version of 
ai-utils
is being used, rather than the git version.
to fix it, try replacing 
ai-utils
with 
ai-utils @ <git+ssh://git@bitbucket.org/unumed/ai_utils.git@v0.5.1>
(I am not sure if uv supports this as well)
bentoml and uv are different tools, the configurations for one may not be read by the other because uv is far from a “built-in standard” package manager
t
Yes, that's it!
Copy code
dependencies = [
    "ai-utils @ <git+ssh://git@bitbucket.org/unumed/ai_utils.git@v0.5.1>",
...

[tool.hatch.metadata]
allow-direct-references = true
This did the trick! Thank you so much
@Frost Ming I've updated to the latest version and something has changed, the private repo can no longer be installed properly. I've tried with this in the project dependencies:
Copy code
dependencies = [
    "ai-utils @ <git+ssh://git@bitbucket.org/unumed/ai_utils.git@v0.5.1>",
...
as well as this in the build.python section (in two separate attempts):
Copy code
[tool.bentoml.build.python]
extra_index_url = ["<https://download.pytorch.org/whl/cpu>"]
pip_args = "--no-deps --no-cache-dir"
lock_packages = false
packages = ["<git+ssh://git@bitbucket.org/unumed/ai_utils.git@v0.5.1>"]
I run 
bentoml build
and then 
bentoml containerize anomaly-detection  --opt ssh=default
after having my ssh client alive and with the proper ssh key added. Am I doing something wrong? For it to have git and ssh available, ive added:
Copy code
[tool.bentoml.build.docker]
system_packages = ["git", "openssh-client"]
But when I build with docker myself, I typically have to add something like this to the dockerfile:
Copy code
# Configure SSH for Bitbucket
RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh
RUN ssh-keyscan bitbucket.org > /root/.ssh/known_hosts

# Install project dependencies using uv
RUN --mount=type=ssh uv sync --all-extras
Edit: oh, v2 doesnt support wheels I'm about to try building it as a wheel and installing it that way instead
It works when I have it as a wheel, however it then uses bento v1 and not bento v2
2 Views
Open in Slack
PreviousNext
Powered by