This message was deleted.

I think I’ve made a mistake in my code, I will try it again.

Hmmm @Rennan Valadares can you tell me more about your onnx model and use cases?

I believe the ORTModelForFeatureExtraction is based on a ABC interface that huggingface provide, which means that we are not currently supporting this with

bentoml.onnx

.

bentoml.onnx

currently only support ONNX model type, which is a subclass of

onnx.ModelProto

. We should add support for more extensive onnxruntime -based model 🤔

@Sean any recommendations right now until we have a onnx suppory?

You are 100% correct @Aaron Pham. That is exactly my use case.

@Rennan Valadares I’d recommend supporting the ORT model through a custom runner. We will look into adding support for this model type. You’re welcome to contribute

ORTModelForFeatureExtraction

support in

bentoml.onnx

as well. @Aaron Pham do you think this should be added under

bentoml.onnx

?

I think we should have a more robust design for supporting different onnxruntime. I don’t think we should add this into

bentoml.onnx

Thanks Sean I will give it a try with a customer runner. How can I contribute ?

@Rennan Valadares once you have a custom runner impl, could you please share with us?