cyh0530
07/30/2025, 9:54 PMTabs [Promptfoo]
08/01/2025, 7:36 PMWill [Promptfoo]
08/01/2025, 7:52 PMcyh0530
08/02/2025, 1:13 AMWill [Promptfoo]
08/04/2025, 1:02 PMdefaultTest block if you have any assertions which should be applied to all messages, and then use the echo providerWill [Promptfoo]
08/04/2025, 1:03 PMWill [Promptfoo]
08/04/2025, 1:03 PMprompts block, you could do something like '{{message}}' and then within your test cases use vars.messagecyh0530
08/04/2025, 5:00 PMWill [Promptfoo]
08/04/2025, 5:04 PMcyh0530
08/04/2025, 5:15 PMWill [Promptfoo]
08/04/2025, 5:46 PMprompts:
- "{{ messages | dump }}"
providers:
- echo
defaultTest:
assert:
- type: model-graded-closedqa
value: The assistant message does not include special punctuation, such as asterisks.
- type: model-graded-closedqa
value: The assistant does not try to finish incomplete sentences for the user.
- type: model-graded-closedqa
value: The assistant does not ask the same question multiple times.
tests:
- vars:
# See https://www.promptfoo.dev/docs/configuration/chat/
messages:
- role: system
content: Respond as a pirate
- role: user
content: Who founded Facebook?
- role: assistant
content: Mark Zuckerberg
- role: user
content: Did he found any other companies?cyh0530
08/04/2025, 5:59 PMWill [Promptfoo]
08/04/2025, 6:04 PM