This message was deleted.

What environment are you running locally, Docker, conda? If you have GPU available locally, an simple step to try is to build an image with

bentoml containerize

and

docker run

the image locally.

was working locally with `bentoml serve`tho I didnt setup anything to support GPU...the tutorial said in the next article he would show how to add GPU support, but its not written yet =X haha...all of the docs are docker config stuff...does the bentoml command just wrap hitting docker?

There was a confusion in the article I think, the “Deploy The Stable Diffusion Bento To EC2” section discusses how to leverage GPU on EC2.

Ahh thanks @Sean I am seeing that now...is there no way to use GPU when running locally in dev?

There is. If you have GPU available locally, the easiest is to containerize and run with docker locally.

gotcha, ty!