<!channel> Hi everyone! We are excited to announce the release of BentoML 1.3! Following the feedback received since the launch of 1.2 earlier this year, we are introducing a host of new features and enhancements in 1.3. Below are the key highlights of 1.3 and stay tuned for an upcoming blog post, where we'll provide a detailed exploration of the new features and the driving forces behind the development. 🕙 Implemented BentoML task execution ◦ Introduced the

@bentoml.task

decorator to set a task endpoint for executing long-running workloads (such as batch processing or video generation). ◦ Added the

.submit()

method to both the sync and async clients, which can submit task inputs via the task endpoint and dedicated worker processes constantly monitor task queues for new work to perform. ◦ Full compatibility with BentoCloud to run Bentos defined with task endpoints. ◦ See the Services and Clients doc with examples of a Service API by initializing a long running task in the Service constructor, creating clients to call the endpoint, and retrieving task status. 🚀 Optimized the build cache to accelerate the build process ◦ Enhanced build speed for

bentoml build

&

containerize

through pre-installed large packages like

torch

◦ Switch to

uv

as the installer and resolver, replacing

pip

🔨 Supported concurrency-based autoscaling on BentoCloud ◦ Added the

concurrency

configuration to the

@bentoml.service

decorator to set the ideal number of simultaneous requests a Service is designed to handle. ◦ Added the

external_queue

configuration to the

@bentoml.service

decorator to queue excess requests until they can be processed within the defined

concurrency

limits. ◦ See the documentation to configure concurrency and external queue. 🔒 Secure data handling with secrets in BentoCloud ◦ You can now create and manage credentials, such as HuggingFace tokens and AWS secrets, securely on BentoCloud and easily apply them across multiple Deployments. ◦ Added secret subcommands to the BentoML CLI for secret management. Run

bentoml secret -h

to learn more. 🗒️ Added streamed logs for Bento image deployment ◦ Easier to troubleshoot build issues and enable faster development iterations 🙏 Thank you for your continued support! Feel free to try 1.3 now!

What a coincidence! I was planning to migrate to 1.2 this afternoon. Seems that I will migrate to 1.3 then 🥳