https://promptfoo.dev/ logo
Given n true answers to the prompt, I would like t...
# ❓|questions
c
Given n true answers to the prompt, I would like to apply the validation logics like promptfloo. In practice it should not make the calls, but apply the validation rules on the answers that I already have (I also have the related parameters) in order to understand that that prompt does not satisfy and then proceed with the changes and tests according to promptflow. At the moment I understand I can only do it if I have temperature = 0 and re-executing the calls to have the same answers that I have already obtained in production
m
Hi @chida82 you can use the
echo
provider to use promptfoo on previously generated llm outputs If you share an example of your config I can help more.
c
Let me study it. Thanks!
In documuntation where is echo? i need use javascript or other script to use custom search in file o db?
i
Echo isn't documented yet actually. But it's just a custom provider that outputs its input. If you need something more complex than that, e.g retrieving from db, then I recommend using a custom provider