https://promptfoo.dev/ logo
That's a great question <@689640780222038070>. The...
# 💬|general
u
That's a great question @jf1384. There's a built-in latency assertion type, no need for a JavaScript assertion. Here's the correct syntax:
Copy code
tests:
    - vars:
        input: "Hello"
      assert:
        - type: latency
          threshold: 5000  # max milliseconds allowed
This will pass/fail based on whether the response time is under the threshold. Make sure to use --no-cache since latency isn't tracked for cached results. For token usage, you can use a JavaScript assertion:
Copy code
assert:
    - type: javascript
      value: "context.providerResponse.tokenUsage.total < 1000"
There's also a built-in cost assertion if you want to cap your spend per call:
Copy code
assert:
      - type: cost
        threshold: 0.01  # max USD
        metric: cost_usd
Regarding the UI metrics dropdown, that filters by custom named scores, not latency/tokens directly. Latency and token usage are shown in the summary header per provider column. If you want latency to appear as a filterable metric in the dropdown, add the metric field:
Copy code
assert:
    - type: latency
      threshold: 5000
      metric: latency_ms
Docs: https://www.promptfoo.dev/docs/configuration/expected-outputs/#deterministic-eval-metrics
2 Views