https://promptfoo.dev/ logo
Validate json-response
# ❓|questions
c
hello i recently started using promptfoo and wanted to use it for validating if LLMs responses are proper json. my expected json is:
Copy code
{
  "fields": [
    {
      "code": "NL",
      "value": [
        1,
        2,
        3
      ]
    },
    {
      "code": "KL",
      "value": true
    }
  ]
}
so i want to validate if NL has 1,2,3 values and if KL is true. but still i'm a bit confused, i managed to create test variables that are separate test-cases, but i want validate every response as well
Copy code
prompts:
  - file://prompt_1.yaml
providers:
  - id: openai:gpt-4o
    config:
      temperature: 1
      response_format:
        type: json_object
  - id: openai:gpt-4o-mini
    config:
      temperature: 1
      response_format:
        type: json_object
scenarios: []
tests:
  - file://permutations.csv
  - assert: #this is wrong
      - type: contains-json
        value:
          fields:
            - code: NL
              value: 1
i
if I understand correctly, you should put the assert on
defaultTest
if you want it to apply to everything
Copy code
yaml
defaultTest:
  assert:
    - type: contains-json
      value: ...
c
yup, problem solved got it from docs 😄