raghadn3
03/24/2024, 6:00 AMraghadn3
03/24/2024, 6:53 AMian
03/24/2024, 8:43 PMyaml
prompts: [prompt1.txt, prompt2.txt]
providers: [openai:gpt-3.5-turbo, ...]
tests: tests.csv
Your prompt might look like:
Translate this text to {{language}}:
{{document}}
and the CSV format could looks like:
language,document,__expected1,__expected2
French,"Greetings",contains:bonjour,similar(0.5):Hello world
Personally I find it a bit easier to just include it all in the YAML:
yaml
prompts: [prompt1.txt, prompt2.txt]
providers: [openai:gpt-3.5-turbo, ...]
tests:
- vars:
language: French
document: "..."
assert:
# Semantic similarity
- type: similar
threshold: 0.5
value: "Hello world"
# String contains
- type: contains
value: "Bonjour..."