Hi guys, your framework's support for out of the box batched inference support is very good. That's one of the main reasons I am exploring the sdk
Speaking of the same how would I deploy a CLIP model in sagemaker inference endpoint
My current flow is as follows
Input JSON :
{
"type": "text" or "image"
"inputs" : image or text content
}
depending upon the input type I perform the inference and return the embedding
This is because aws sagemaker inference endpoint exposes only one endpoint
How can I do a similar flow with batched inferencing in bentoml. I am asking this as in your clip example you created 3 endpoints, but sagemaker inference endpoint doesnt support more than one endpoint