https://promptfoo.dev/ logo
Can I create a "custom" contains function when rea...
# ❓|questions
f
Hi! Been banging my head against the wall for a bit and was hoping someone can help me figuring this out. I have a usecase as follows: I have a csv file on the following format: Variable1,Variable2, expectedOutput1, expectedOutput2 I have a prompt which basically is: " Please do bla bla with {{variable1}} and bla bla with {{variable2}}" I want to verify that the response contains expectedOutput1 and expectedOutput2. And that expectedOutput1 comes before expectedOutput2. Ideally it should work no matter what it says in between expectedOutput1 and expectedOutput2, but I might be able to figure out the exact string. The closest I have come to a solution is to manually change the csv so that the each expectedOutput has a contain statement like this: Variable1,Variable2, fn.contains(expectedOutput1), fn.contains(expectedOutput2) But this forces me to update the csv I receive which is a bit tricky, and it does not verify the correct order. Thx beforehand on any help I can receive
i
one way to do this might be with a regex pattern:
Copy code
yaml
assert:
  - type: regex  
    value: "expectedOutput1.*expectedOutput2"
(or the equivalent in CSV)
b
or js script or python?
t
@beautifulpython what do you mean?
2 Views