This message was deleted.

Requests to the service can arrive at different time. The batching logic will create the most optimal batch within the boundaries of

max_latency

and

max_batch_size

.