Hello, I have a question regarding the batchable API provided by BentoML. When configuring an API as batchable like this:

python


@bentoml.api(batchable=True, batch_dim=0, max_batch_size=10, max_latency_ms=5000)

Does the batch execution require at least one of the following conditions to be met? 1. Maximum batch size (

max_batch_size

) 2. Maximum latency (in milliseconds) until the maximum batch size is reached (

max_latency_ms

) Or, can the batch execute even if neither of these conditions is met? If the batch can execute without meeting either condition, is this behavior due to BentoML's internal mechanism? Thank you. I look forward to your response.