@channel Hi everyone! We are thrilled to announce the release of OpenLLM 0.6 🚀, which marks a significant shift in our project's philosophy. This release introduces breaking changes to the codebase, reflecting our renewed focus on streamlining cloud deployment for LLMs. In the previous releases, our goal was to provide users with the ability to fully customize their LLM deployment. However, we realized that the customization support in OpenLLM led to scope creep, deviating from our core focus on making LLM deployment simple. With the rise of open-source LLMs and the growing emphasis on LLM-focused application development, we have decided to concentrate on what OpenLLM does best - simplifying LLM deployment. As such, we have completely revamped the architecture to make OpenLLM a tool that simplifies running LLMs as an API endpoint, prioritizing ease of use and performance. This means that 0.6 breaks away from many of the old Python APIs provided in 0.5, emphasizing itself as an easy-to-use CLI tool with cross-platform compatibility for users to deploy open-source LLMs. Some of the coolest features and capabilities include: • Broad LLM support: Support a wide variety of open-source LLMs, including those fine-tuned with your own data or enhanced through advanced quantization. • OpenAI-compatible endpoints: Serve your LLMs with endpoints fully compatible with OpenAI standards, ensuring ease of integration. • Enhanced decoding speed: Accelerated LLM decoding powered by the state-of-the-art inference backend. • Interactive chat UI: Chat with different models with a built-in chat user interface. • Enterprise-grade cloud deployment: Optionally to deploy to BentoCloud with a single command for an enterprise-grade LLM API endpoint. To learn more, visit the OpenLLM repository. 🤝 We invite you to explore the new release, provide feedback, and join us in our mission to make cloud deployment of LLMs accessible and efficient for everyone. 🙏 Thank you for your continued support and trust in OpenLLM. We look forward to seeing the incredible applications you will build with the tool!