This message was deleted BentoML #ask-for-help

Join Slack

This message was deleted.

# ask-for-help

Slackbot

04/14/2023, 12:59 PM

This message was deleted.

larme (shenyang)

04/14/2023, 9:22 PM

Hi,

bentoml.diffusers

is to simplify the usage of diffusers in BentoML. It has not much difference with using diffusers by itself. Though

bentoml.diffusers

will try to utilize

xformers

to accelerate the computation, you can also use xformers if you use diffusers yourself

🍱 2

Self

04/17/2023, 1:24 PM

I see thanks! Currently

bentoml.diffusers

creates a bug for me (opened an issue for that), but using diffusers with a custom bentoml runner works

Self

04/17/2023, 1:30 PM

Is it known if Stable Diffusion images can be batched? In Diffusers it should be possible right ?

Copy code

@bentoml.Runnable.method(batchable=False, batch_dim=0)
        def txt2img(self, input_data):

If I set

batchable=True

the generation does not seem to work. Is the batch dimension a different one than 0 ?

larme (shenyang)

04/17/2023, 3:42 PM

Hi Self, I think the batching in Stable Diffusion need a lot of settings (width etc.) to be the same inside a batch. What BentoML's batching do is actually merging single requests from different users to a batch and feed the batch to model. In sd's context I think it may be not helpful to turn on batching.

larme (shenyang)

04/17/2023, 3:42 PM

And what is the issue you open? I haven't found it yet.

Self

04/18/2023, 2:45 PM

I see you have replied on the issue, thanks! The batching makes sense and would likely be independent of the API batching of bentoml (thinking of a GPU with lots of memory to generate e.g. 2 or 4 images in a single batch)

Open in Slack

Previous Next