dadatm
09/20/2024, 9:11 AMtests:
- vars:
query: 'What are the number of visits?'
assert:
# - type: is-json
- type: contains-json
value:
required:
- read_metric_value
properties:
read_metric_value:
type: boolean
# value: true # This doesn't work
Basically I want to check that the response matches something like this:
{
"read_metric_value": true
}michaelmichaelmichael
09/20/2024, 2:19 PMproviders:
- echo
prompts: '{ "read_metric_value": true }'
tests:
- vars:
query: 'What are the number of visits?'
assert:
- type: contains-json
value:
type: object
required:
- read_metric_value
properties:
read_metric_value:
type: boolean
enum: [true]
If you change either true to false it will fail.
Check out https://json-schema.org/learn/getting-started-step-by-step for a quick tour of JSON schemas (or ask an LLM - I asked Claude to correct your schema because I didn't know the best answer). Hope this helps!dadatm
09/23/2024, 9:14 AM