https://promptfoo.dev/ logo
promptfoo-action questions
# ❓|questions
j
Hello! I am integrating the
promptfoo-action
([link](https://github.com/promptfoo/promptfoo-action)), and have a few questions: - Why does
restore-keys
not use
v1
at the end? - With the line "This step will actually run the before/after evaluation", what is a before/after evaluation? - If I have `prompts/config1.yaml`/`prompts1.txt` and `prompts/config2.yaml`/`prompts2.txt`, should I have two steps invoking
typpo/promptfoo-action@v1
, one per each config?
i
Hi James! -
restore-keys
is a prefix, so it will match any version - "before/after" just describes the webview. One column for the original prompt, one column for the updated prompt - Yeah, each config will have to be run in a separate step
j
Thanks! BTW your real life mankey and animorphs profile combo is genius, cracked me up
i
😅
j
Do you think CI should set
PROMPTFOO_DISABLE_TELEMETRY=1
? I am wondering if you care that CI agents will bias the telemetry collection numbers
i
hmm, that could make sense
honestly - not terribly concerned about the accuracy of those numbers. I added it early on so I could tell whether anyone was actually using the project
j
One other question is, do you get GitHub notifications for
promptfoo-actions
? I hit an issue where it fails to run
git fetch origin main
, opened a bug report there
i
👀 I'll take a look