BentoML

Novice followup question: Does it mean that I don't need to introduce a queue ( SQS ) for any Bentos I use? That would be amazing

Guess this is followup to the question from <@U06MDM0KY64>

We are adding a “@bentoml.task” in the upcoming release which will have a built in message queue for long running inference task. The “@bentoml.api” endpoints will queue up requests but in many cases you’d still need an external queue for handling burst traffic or using queue/concurrency based auto scaling 

Btw BentoCloud provides a scalable external queue and concurrency based auto scaling out of the box

<https://docs.bentoml.com/en/latest/bentocloud/how-tos/autoscaling.html|https://docs.bentoml.com/en/latest/bentocloud/how-tos/autoscaling.html>

Thanks <@UKB4CLKP1>. Will get to BentoCloud soon. I love your product. Just need to go through the motions and get my MVP up and running. Then with some VC backing, it should be easy to convince to get onto a paid model as you guys deserve