https://promptfoo.dev/ logo
Assertion Template with JSON config
# ❓|questions
j
Is it possible to use assertion templates while using a promptfooconfig.json? I've tried it and can't get it right. Here is what I've done so far:
Copy code
json
{
    "assertionTemplates": {
        "containsMentalHealth": {
            "type": "javascript",
            "value": "output.toLowerCase().includes('mental health')",
            "provider": "openai:gpt-3.5-turbo-0301"
        }
    },
    "prompts": [
        "prompts.txt"
    ],
    "providers": [
        "openai:gpt-3.5-turbo-0301"    
    ],
    "tests": [
        {
            "vars": {
                "question": "Tell me about the benefits of exercise."
            },
            "assert": [
                {
                    "$ref": "#/assertionTemplates/containsMentalHealth"
                }
            ]
        }
    ]
}
i
hi jonas, this was only supported for YAML configs - so I just made a change to enable this for
.json
configs as well https://github.com/promptfoo/promptfoo/commit/c32f9b051a51ee6e1ee08738e0921b4e05a5c23d It will be included in the next release!
j
@ian Thanks!
2 Views