a.u.s.t.y.n.
01/17/2024, 4:13 PMian
01/17/2024, 4:45 PMassert object of each test case takes a list of assertions. Can you provide multiple assertions there? That will use a single LLM call, but run multiple checks.a.u.s.t.y.n.
01/17/2024, 5:42 PMa.u.s.t.y.n.
01/23/2024, 9:55 PMassert:
- type: equals
value: SQL
key: category
- type: contains
value: 101
key: response
If this were to be implemented it would help our team out a whole lot and I suspect it would be used by others as well. Thanks for looking at this for us!a.u.s.t.y.n.
01/29/2024, 7:32 PMian
01/30/2024, 3:21 PMpostprocess](https://promptfoo.dev/docs/configuration/guide/#postprocessing) directive for assertions. It would look like this:
yaml
assert:
- type: equals
value: SQL
postprocess: output.category
This is consistent with other supported behavior to e.g. pick keys out of JSON objects. What do you think?
Also, just to make sure that you're aware, you can achieve similar results with this:
yaml
assert:
- type: javascript
value: output.category === 'SQL'
which is not too awful imo..a.u.s.t.y.n.
01/30/2024, 4:46 PMian
01/30/2024, 10:31 PMpostprocess is a small lift. Let me see if I can get it done quickly 🙂ian
01/31/2024, 12:52 AMian
01/31/2024, 12:52 AMa.u.s.t.y.n.
01/31/2024, 5:46 PMa.u.s.t.y.n.
02/02/2024, 3:36 PM