This message was deleted.

This message was deleted.

im pretty clueless as im unable to understand what exactly I should be doing.

Can someone help me out?

This message was deleted.

This message was deleted.

This message was deleted.

Hi <!channel>! We have just released BentoML 1.0.0rc2 with an exciting lineup of improvements. Checkout it out with the following command!

$ pip install -U bentoml --pre

• Standardized logging configuration and improved logging performance. ◦ If imported as a library, BentoML will no longer configure logging explicitly and will respect the logging configuration of the importing Python process. To customize BentoML logging as a library, configurations can be added for the

bentoml

logger.

formatters:
  ...
handlers:
  ...
loggers:
  ...
  bentoml:
    handlers: [...]
    level: INFO
    ...

◦ If started as a server, BentoML will continue to configure logging format and output to

stdout

at

INFO

level. All third party libraries will be configured to log at the

WARNING

level. • Added LightGBM framework support. 🎉 • Updated model and bento creation timestamps CLI display to use the local timezone for better use experience, while timestamps in metadata will remain in the UTC timezone. • Improved the reliability of bento build with advanced options including base_image and dockerfile_template. Beside all the exciting product work, we also started a blog at modelserving.com sharing our learnings gained from building BentoML and supporting the MLOps community. Checkout our latest blog Breaking up with Flask &amp; FastAPI: Why ML model serving requires a specialized framework (share your thoughts with us on our LinkedIn post). Lastly, a big shoutout to @Mike Kuhlen for adding the LightGBM framework support. 🥂

This message was deleted.

Hi <!channel>, we have just released BentoML

1.0.0rc3

with a number of highly anticipated features and improvements. Check it out with the following command!

$ pip install -U bentoml --pre

⚠️ BentoML will release the official

1.0.0

version next week and remove the need to use

--pre

tag to install BentoML versions after

1.0.0

. If you wish to stay on the

0.13.1

LTS version, please lock the dependency with

bentoml==0.13.1

. • Added support for framework runners in the following ML frameworks. ◦ fast.ai ◦ CatBoost ◦ ONNX • Added support for Huggingface Transformers custom pipelines. • Fixed a logging issue causing the api_server and runners to not generate error logs. • Optimized Tensorflow inference procedure. • Improved resource request configuration for runners. ◦ Resource request can be now configured in the BentoML configuration. If unspecified, runners will be scheduled to best utilized the available system resources.

runners:
  resources:
    cpu: 8.0
    <http://nvidia.com/gpu|nvidia.com/gpu>: 4.0

◦ Updated the API for custom runners to declare the types of supported resources.

import bentoml

class MyRunnable(bentoml.Runnable):
  SUPPORTS_CPU_MULTI_THREADING = True  # Deprecated SUPPORT_CPU_MULTI_THREADING 
  SUPPORTED_RESOURCES = ("<http://nvidia.com/gpu|nvidia.com/gpu>", "cpu")  # Deprecated SUPPORT_NVIDIA_GPU
  ...

  my_runner = bentoml.Runner(
    MyRunnable,
    runnable_init_params={"foo": foo, "bar": bar},
    name="custom_runner_name",
    ...
)

◦ Deprecated the API for specifying resources from the framework to_runner() and custom Runner APIs. For better flexibility at runtime, it is recommended to specifying resources through configuration.

Friendly/Gentle/Public Reminder that if you are using 0.13: “pip install bentoml” will break if you have this as part of your deployment pipeline. Please remember to update to “pip install bentoml==0.13” in order to ensure the health of your pipeline!

This message was deleted.

This message was deleted.

Announcing our official 1.0 release on hackernews!

🍱 Hi <!channel>, we have just released BentoML v1.0.2 with a number of features and bug fixes requested by the community. • Added support for custom model versions, e.g.

bentoml.tensorflow.save_model("model_name:1.2.4", model)

. • Fixed PyTorch Runner payload serialization issue due to tensor not on CPU.

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first

• Fixed Transformers GPU device assignment due to kwargs handling. • Fixed excessive Runner thread spawning issue under high load. • Fixed PyTorch Runner inference error due to saving tensor during inference mode.

RuntimeError: Inference tensors cannot be saved for backward. To work around you can make a clone to get a normal tensor and use it in autograd.

• Fixed Keras Runner error when the input has only a single element. • Deprecated the

validate_json

option in JSON IO descriptor and recommended specifying validation logic natively in the Pydantic model. 🎨 We added an examples directory and in it you will find interesting sample projects demonstrating various applications of BentoML. We welcome your contribution if you have a project idea and would like to share with the community. 💡 We continue to update the documentation on every release to help our users unlock the full power of BentoML. • Did you know BentoML service supports mounting and calling runners from custom FastAPI and Flask apps? • Did you know IO descriptor supports input and output validation of schema, shape, and data types?

🍱 Hi <!channel>, BentoML v1.0.3 release is here and has brought a list of performance and feature improvement. • Improved Runner IO performance by enhancing the underlying serialization and deserialization, especially in models with large input and output sizes. Our image input benchmark showed a 100% throughput improvement (see comparison in attachments 1 & 2). • Added support for specifying URLs to exclude from tracing. • Added support custom components for OpenAPI generation (see attachment 3). 🙌 We continue to receive great engagement and support from the BentoML community. • Shout out to Ben Kessler for helping benchmarking performance. • Shout out to Jiew Peng Lim for adding the support for configuring URLs to exclude from tracing. • Shout out to Susana Bouchardet for add the support for JSON IO Descriptor to return empty response body. • Thanks to Keming and mplk for contributing their first PRs in BentoML.

🍱 Hello <!channel>, You asked for more conversations about MLOps best practices. We are taking action on that. 🐝🐝🐝 🎤 We are hosting monthly #ask-me-anything sessions to talk with our prominent community members and industry leaders. Stay tuned for the August lineup. 🥘 Join #ml-learning-potluck to learn the latest industry news and BentoML tips. 🌍🌎🌏 We created 3 language channels for chatting about MLOps in your preferred language. #português-portuguese #한국어-korean #中文-chinese If you want to add your preferred language channel or any feedback. Feel free to DM me.

Hello everyone! Join us on August 30th, 9 am-10 am PST(1800 1900 CEST) for our Ask Me Anything Session! Our guest is @Gabriel Bayomi. He is the CEO and co-founder of http://unbox.ai Gabe is passionate about Natural Language Processing, Machine Learning, and Computational Social Science. Previously MSc Computer Science @ CMU and Ignite @ Stanford GSB. Fellow at Fundação Estudar. Unbox is a testing and debugging platform for machine learning models. Unbox is founded by 3 Apple Siri engineers and is part of the YC S21 batch. As one of the early BentoML users, Unbox is able to help its customers to debug their models with BentoML For the Ask Me Anything session, join us and ask questions about: • Model evaluation • ML explainability • Error Analysis • Data generation Feel free to share this with those who might be interested in it! React to this message to get invited to the #C03U2PT7UUQ channel

<!channel> we have our Ask-Me-Anything session with Gabriel next week. His company, Unbox, helps ML partitioners to debug their models. BentoML helps unbox to spin up services quickly for their customers to test, debug and improve models. You can ask Gabe questions from how to debug model and error analysis to how are they using BentoML in production. We will give out swags to the top 3 questions with the most emoji reactions! React to this message to get an invite to the #ask-me-anything channel

🍱 <!channel>, BentoML v1.0.4 is here! • Added support for explicit GPU mapping for runners. In addition to specifying the number of GPU devices allocated to a runner, we can map a list of device IDs directly to a runner through configuration.

runners:
  iris_clf_1:
    resources:
      <http://nvidia.com/gpu|nvidia.com/gpu>: [2, 4] # Map device 2 and 4 to iris_clf_1 runner
  iris_clf_2:
    resources:
      <http://nvidia.com/gpu|nvidia.com/gpu>: [1, 3] # Map device 1 and 3 to iris_clf_2 runner

• Added SSL support for API server through both CLI and configuration.

--ssl-certfile TEXT          SSL certificate file
  --ssl-keyfile TEXT           SSL key file
  --ssl-keyfile-password TEXT  SSL keyfile password
  --ssl-version INTEGER        SSL version to use (see stdlib 'ssl' module)
  --ssl-cert-reqs INTEGER      Whether client certificate is required (see stdlib 'ssl' module)
  --ssl-ca-certs TEXT          CA certificates file
  --ssl-ciphers TEXT           Ciphers to use (see stdlib 'ssl' module)

• Added adaptive batching size histogram metrics,

BENTOML_{runner}_{method}_adaptive_batch_size_bucket

, for observability of batching mechanism details. • Added support OpenTelemetry OTLP exporter for tracing and configures the OpenTelemetry resource automatically if user has not explicitly configured it through environment variables. Upgraded OpenTelemetry python packages to version

0.33b0

. • Added support for saving

external_modules

alongside with models in the

save_model

API. Saving external Python modules is useful for models with external dependencies, such as tokenizers, preprocessors, and configurations. • Enhanced Swagger UI to include additional documentation and helper links. 💡 We continue to update the documentation on every release to help our users unlock the full power of BentoML. • Checkout the adaptive batching documentation on how to leverage batching to improve inference latency and efficiency. • Checkout the runner configuration documentation on how to customize resource allocation for runners at run time. 🙌 We continue to receive great engagement and support from the BentoML community. • Shout out to @Steve T for their contribution on adding SSL support. • Shout out to @Daniel Buades Marcos for their contribution on adding the OTLP exporter. • Shout out to @Phil D'Amore for their contribution on fixing a bug on

import_model

in the MLflow framework.

This message was deleted.

This message was deleted.

This message was deleted.

Hello <!channel> Happy Thursday! We’re excited to chat with @Aparna Dhinakaran on September 29th, 1-2 pm PST/4-5 pm EST for our AMA session on all things MLOps. Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, an ML observability platform. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she contributed to several core ML Infrastructure platforms, including Michaelangelo. She has a bachelor’s from UC Berkeley in Electrical Engineering and Computer Science, where she published research with Berkeley's AI Research group. For this session, join us to ask questions about: • The emerging MLOps toolchain • Productionalizing your research model • Troubleshooting model issues in real-time • Improving your overall model performance Feel free to schedule your questions if you can’t attend the live AMA: (https://slack.com/help/articles/1500012915082-Schedule-messages-to-send-later)

Happy Thursday everyone! It's one week away from the AMA with Aparna on MLOps toolchain, troubleshooting models, and how to improve overall model performance. If you can't make it to the AMA session, be sure to use the schedule your question with slack's send it later feature.

Hi <!channel>, happy Friday! Reminder to fill out our community survey if you’d like to provide feedback for the project and help define the roadmap for BentoML: https://forms.gle/ZMZeC52eZQnFUj1M8. We will also be sending out swags to those who participated!

This message was deleted.

Happy Wednesday everyone We are 1 day away from our AMA with Aparna. She is the Co-Founder and Chief Product Officer at Arize AI, an ML observability platform. Aparna was recently named to the Forbes 30 Under 30. Before Arize, She was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she contributed to several core ML Infrastructure platforms, including Michaelangelo. She has a bachelor’s from UC Berkeley in Electrical Engineering and Computer Science, where she published research with Berkeley’s AI Research group. Join us tomorrow (9/29, 1pm PST) to ask questions about: • The emerging MLOps toolchain • Productionalizing your research model • Troubleshooting model issues in real-time • Improving your overall model performance

This message was deleted.

This message was deleted.