asterwork
01/15/2024, 7:11 PMpy
with subprocess.Popen(
["promptfoo", "eval", "-o", "output.json", "--no-table"], stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True, encoding="utf-8", shell=True, bufsize=1, universal_newlines=True) as eval_result:
for line in eval_result.stdout:
print(line, end='')
I've tried variations as well, but all I'm seeing is the end result.
cmd
Writing output to output.json
======================================================================
✔ Evaluation complete.
Run promptfoo view to use the local web viewer
Run promptfoo share to create a shareable URL
Run promptfoo feedback to share feedback with the developers of this tool
======================================================================
Successes: 3
Failures: 0ian
01/16/2024, 7:43 PMpy
import subprocess
cmd = ['promptfoo', 'eval', '--no-table']
subprocess.run(cmd)asterwork
01/18/2024, 9:46 PMcapture_output=True or piping both the stdout and stderr was the culprit for not showing the output (i needed to capture stdout for later)
so instead of
py
subprocess.run(
["promptfoo", "eval", "-o", "temp_output_holder.json", "--no-table"], capture_output=True,
text=True, encoding="utf-8", shell=True)
i changed it to
py
subprocess.run(
["promptfoo", "eval", "-o", "temp_output_holder.json", "--no-table"], stdout=subprocess.PIPE,
text=True, encoding="utf-8", shell=True)