Powered by
llm-rubric and maxConcurrency
# ❓|questions
m
manu3b
03/18/2024, 10:27 AM
Looks like call to the provider by llm-rubric is serialized, which erases benefits of maxConcurrency. Am I right?
i
ian
03/18/2024, 4:18 PM
Hmm each of the eval steps should still run in their own thread, including assertions. The only exception to this is the
select-best
assertion, which has to compare between multiple outputs.
m
manu3b
03/19/2024, 2:58 PM
Oops, I was playing with maxConcurrency at yaml level, but looks like it is available at command line one (-j). Thx!
Previous
Next