jimmy_soup
04/01/2024, 3:41 PMian
04/01/2024, 4:03 PMyaml
tests:
- description: 'unparsed json tests'
assert:
- type: is-json
# ...
- description: 'parsed json tests'
options:
transform: JSON.parse(output)
assert:
- type: llm-rubric
value: # ...
but you want to avoid repeating everything for each set of `var`s.
Maybe it's not ideal, but you could use defaultTest and put the transform on each assertion.
yaml
defaultTest:
assert:
# JSON assertions
- type: is-json
# ...
# Parsed assertions
- type: llm-rubric
transform: JSON.parse(output)
value: # ...
You'll have to repeat the transform for each parsed assertion, but at least you won't be repeating it in every single testcase.
Hope I understood correctly, and that this helps.jimmy_soup
04/01/2024, 4:07 PMjimmy_soup
04/01/2024, 4:07 PMjimmy_soup
04/01/2024, 5:23 PM