Hello, I created an experiment that call python script to load LLM.
The global experiment contain a lot of experiment : 8k
I desactivated the cache (because each one is unique)
But it take too long, and I would like to know if there is a way to access the result that are already been computed ?
An optimisation possible in promptfoo to be faster on that point could be the possibility to batch all the prompt (with there variable) for the same model so the model is loaded only once and not each time
thanks in advance for your response
i
ian
05/23/2024, 5:59 AM
Have you tried to adjust the concurrency?
For example:
Copy code
promptfoo eval --max-concurrency 16
The default is purposely low because most providers rate limit user requests
a
AlexisIMBERT
05/23/2024, 2:00 PM
Hi thank you for your answer,
I have to keep it to 1 because all models are load on the computer (i don't use any API) so I can't load 2 models at the same time
i
ian
05/25/2024, 6:42 PM
hmm, it kind of seems like that's the limiting factor, not lack of batch requests!