https://promptfoo.dev/ logo
[SOLVED] Only single assertion check with csv test...
# ❓|questions
k
Hi! I'm testing the tests from CSV (which contains 2 questions). ** CSV structure __description question expectedAnswer gradedValue I would like each of the question to run through 2 assertions, according to my configuration. But eval always returns only a single assertion by question. I tried different approaches (with templates, default test) no effects. ** YAML: providers: - openai:assistant:asst_xxxxxxxxxxx prompts: - id: id 1 raw: {{question}} tests: file://..\test_assertions.csv testCaseTemplate: assert: - type: factuality value: "{{expectedAnswer}}" - type: model-graded-closedqa value: "{{gradedValue}}" defaultTest: options: provider: openai:gpt-4o-mini #default grader for tests What should be the yaml config to make it working? [████████████████████████████████████████] 100% | ETA: 0s | 1/1 | openai:asst_xxxxxx" question=C [████████████████████████████████████████] 100% | ETA: 0s | 1/1 | openai:asst_xxxxxx" question=C BR Kmet
m
Hi @kmet_09025 can you share your csv (or a redacted version of it)? If you don't want to share it here you can email it to michael@promptfoo.dev
k
Thank you @michaelmichaelmichael ! It works now. The assert block needed to be unindented by one level to align with the options block.
Copy code
defaultTest:
  options:
    provider: openai:gpt-4o-mini # default grader for tests
  assert:
    - type: factuality
      value: "{{expectedAnswer}}"
    - type: model-graded-closedqa
      value: "{{gradedValue}}"
One additional question. the only way I can see the results for both assertions is a "view prompt" icon in Outputs section of eval-web view, correct? On the default view, for the first test (row1), I can see only "1 Fail, 1 Pass" and output from the first assertion.
2 Views