Hi Evryone, I am glad to be joining the BentoML community, I am migrating to bentoML for all production deployments and need help with a service deployment strategy.
1. I have an SKLearn Pipeline Model for Preprocessing and a Catboost Regressor model for prediction.
2. The above inference pipeline is unique to one "market".
Q: How should I deploy the inference pipeline (2 models: Sklearn - Preprocessor + Catboost Regressor) for let's say 30 different markets.
Comments:
• Idk if model composition will help because model size could be upto 5Gb/market and is currently stored in AWS s3 after using a Sagemaker Tuning Job; and therefore will also impact the container size.
• I would prefer to minimize costs and keep the implementation scalable to accommodate for more markets. I will appreciate any help in thread.