This message was deleted.

Hi @Marco Aleixo, thanks for your reply, If I understand correctly, micro-batching is useful in batching multiple single requests. My use case deals with producing prediction for a batch of rows (e.g. a csv with thousands or tens of thousands of rows) asynchronously.

did you check this 🙂 ?