https://promptfoo.dev/ logo
Multiple output files
# ❓|questions
b
Is it possible to output both JSON and HTML for a CI/CD pipeline?
m
Are you using the github action or did you implement your own ci/cd pipeline? When running
promptfoo eval
You can pass multiple file names after -o ex
promptfoo eval -o a.csv a.txt a.json
and it should output all of them
b
Ahhh, I am running a Jenkins CI/CD which I made
m
well -o should do the trick. If it doesn't please let me know and we can look together
b
That works excellent thank you. @michaelmichaelmichael Is there a way to output on an assert by assert basis instead of on a var by var basis? It really doesn't look great from a display perspective if you get 5 asserts passing but the failure entire test fails. Equally, I don't want to repeatedly type out the same var, ideally.
m
Probably! Can you provide a screen shot or a little more context? How are you outputting this? Are you referring to the table or a file format?
b
@michaelmichaelmichael sorry for slow response. I won't be able to screen shot as it cotains PII but I'm basically just getting the terminal output in Jenkins CI/CD. I am then outputting both the HTML and json files for reference using Jenkins Artifacts. Ideally I just want a table which is input variable and then individual assert result. Rather than on a variable by variable basis.
m
@beigesponge please throw some time on my calendar and we can discuss https://cal.com/michael-dangelo/15min