https://promptfoo.dev/ logo
llm-rubric and maxConcurrency
# ❓|questions
m
Looks like call to the provider by llm-rubric is serialized, which erases benefits of maxConcurrency. Am I right?
i
Hmm each of the eval steps should still run in their own thread, including assertions. The only exception to this is the
select-best
assertion, which has to compare between multiple outputs.
m
Oops, I was playing with maxConcurrency at yaml level, but looks like it is available at command line one (-j). Thx!