This message was deleted.

Hey @romit Could you elaborate more about your requirement? You are asking about can bentoml 'hot load' a model that is saved in s3? Is that correct?

Hey @Bo Yes, so while serving I “pack” a model file which is stored in a S3 location

@romit Could you tell me more about your use cases? Currently, BentoML has dynamic load model artifacts on the roadmap. We think this feature will speed up the bento service development workflow and add better support for online learning. For production setting, we recommend use existing blue/green deployment tools to update new models. The devops team can treat model serving the same way as micro-services for deployment with BentoML.

@Bo Got it. My use case is simple, when serving the model via BentoML, I want to pack and load model on the go, i.e I don’t want to create a new version of the class and then deploy. Also loading a model file from remote, would help me make inferencing and training part separate. We have a small team so taking help from devops is a bit tough right now. Is there a guide somewhere that mentions how this happens in conjunction to serving models with BentoML?