https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
l
`bentofile.yaml`:
Copy code
service: "service.py:svc"
include:
  - "service.py"
  - "requirements.txt"
  - "models/v1_4"
  - "configuration.yaml"
python:
  packages:
    - torch
    - transformers
    - diffusers
    - ftfy
docker:
  distro: debian
  cuda_version: "11.6.2"
  env:
    BENTOML_CONFIG: "src/configuration.yaml"
According to NVIDIA docs, cuda 11.6 should work on Ampere architecture
b
Copy code
pip3 install torch torchvision torchaudio --extra-index-url <https://download.pytorch.org/whl/cu116>
Basically, you should install PyTorch with the 
--extra-index-url
l
Yes, I'm trying the latter command. But this would mean that the version of pytorch installed in the container doesn't have the cu116 version
b
Copy code
pip3 install torch torchvision torchaudio --extra-index-url <https://download.pytorch.org/whl/cu116>
l
Is there a way to force bentoml to use a specific 
extra-index-url
when install python packages?
b
yes
l
ahhh
Awesome
b
It's two different things. One is having CUDA 11.6 installed and the other is having PyTorch compiled with CUDA 11.6 support
l
Yep
b
😄
l
Make sense. For a beginner (me, totally), it's really hard to make a head of all the different packages and versions to install hahaha
b
Tell me if it works or not 😄
l
Will do! Trying that now and will let you know
Big thanks!
Seems like it worked! I've only replaced the torch package manually in the container I had and the model ran fine after that. I'm now creating a new docker image, but I'm assuming this will work as well. Thanks again!
🎉 1
b
Yay!
l
As a note, it doesn't seem like using the inline syntax in the 
bentofile.yaml
is working. This doesn't work
Copy code
python:
  packages:
    - "torch==1.12.1 --extra-index-url <https://download.pytorch.org/whl/cu116>"
    - transformers
    - diffusers
    - ftfy
while this works:
Copy code
python:
  packages:
    - torch
    - transformers
    - diffusers
    - ftfy
  extra_index_url:
    - <https://download.pytorch.org/whl/cu116>
👍 1
2 Views
Open in Slack
PreviousNext
Powered by