https://promptfoo.dev/ logo
Running multiple tests + github actions
# ❓|questions
r
Hello! It is my first time using promptfoo (and doing llm work) so I would really appreciate any support :) I am trying to evaluate about 10-15 different prompts each with their own vars and tests (each prompt does completely different things). The end goal is for these evaluations to be automated through github actions. What is the best way to approach this? Should I separate the promptfoo config files for each prompt? If I do, how do I make the github action also run all of the evaluations? Thank you in advance!
d
for github action example
r
Thank you for your reply! i did implement the github action and currently it works for one config file, but i will need to do more than test one (type of) prompt 🤔
d
you will have to do prompfoo eval -c [file1] [file2] [file3]
thats so far I can tell from the doc
2 Views