Hey guys, another bento noob here 🙂 Got an issue with building container locally During

pip install

step

torch

is not dsscoverable by

xformers

I've used https://github.com/bentoml/BentoSDXLTurbo as template And

requirements.txt

there has

torch

so

pip

should resolve all dependencies properly I found related issue at

xformers

github https://github.com/facebookresearch/xformers/issues/740 But recipes there require some manual work, while in my case it's all abstracted by

bentoml

automation So I'm stuck and not sure what would be the best way forward Anybody else experienced that issue?

7.548 Collecting xformers==0.0.23.post1 (from -r /home/bentoml/bento/env/python/requirements.lock.txt (line 89))
7.548   Using cached xformers-0.0.23.post1.tar.gz (3.9 MB)
7.858   Preparing metadata (setup.py): started
8.096   Preparing metadata (setup.py): finished with status 'error'
8.098   error: subprocess-exited-with-error
8.098
8.098   × python setup.py egg_info did not run successfully.
8.098   │ exit code: 1
8.098   ╰─> [6 lines of output]
8.098       Traceback (most recent call last):
8.098         File "<string>", line 2, in <module>
8.098         File "<pip-setuptools-caller>", line 34, in <module>
8.098         File "/tmp/pip-install-jqcxhvpw/xformers_d7e32e5d5a7749b3a32c733445dfade9/setup.py", line 23, in <module>
8.098           import torch
8.098       ModuleNotFoundError: No module named 'torch'
8.098       [end of output]

hey there, are you installing this on a machine with GPU?

oh i didn't realize that, building it on macos on m1 pro, which has gpu, but i guess it's not support out of the box

You can build the project on BentoCloud

Thanks! I didn't realize you don't have dockerize before shipping to bento cloud. Building on the cloud actually worked pretty well, but there is an edge case for

vllm

model. To generate requirements lock file bento needs local install of that package, but it can't be done without CUDA on machine. So far I didn't find a workaround how alternatively generate requirements lock file to continue building on the bento cloud. Related issue at the

vllm

repo https://github.com/vllm-project/vllm/issues/1441

One workaround is to set lock_packages to false in the bentofile.yaml, which allows you to deploy to BentoCloud from a machine that can’t install cuda or vLLM

Hi everyone, I'm also new to BentoML and got the exact same error (which makes no sense). Can I get confirmation that "not finding torch" really means your computer doesn't have a GPU? It's kind-of a poor user experience because I clearly have PyTorch installed:

(py311) $ python

Python 3.11.2 (v3.11.2:878ead1ac1, Feb  7 2023, 10:02:41) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import torch

>>>

It would be nicer if the error when building

xformers

was something like:

GPU not found

rather than

ModuleNotFoundError: No module named 'torch'

In the meantime, I'll try other advice found at https://github.com/facebookresearch/xformers/issues/740 ... for what it's worth, I have a 2020 MBP 13" Intel (not M1), so pretty sure it doesn't have a separate GPU, but I have to still complain about the "error." 🙂 Quick follow-up: I'm getting farther with installing the requirements after running

pip install wheel

Hi there, it requires NVIDIA GPU

Hi, thanks Aaron. Is there an easy way to tell when you do NOT have an nVidia GPU? (Is the error above the only way other than checking your computer's specs, or are there other ways or errors?)