This message was deleted.

Try turning off batching?

If you are using a custom runner class, the runner method should return a list instead of a dict. for ex,

@bentoml.Runnable.method(batchable=True, batch_dim=0)
def predict(self, contents) -> List[dict]:
.
.
.
    return [<list of predictions>]

With this you can make it bachable. Basically for bachable runner's predict method's return type should be List.