https://promptfoo.dev/ logo
GPT-5 series results gives wrong latency and cost=...
# questions
a
When I use any of the gpt-5 models and try to see the metrics, like the latency and cost, the latency for gpt-5-nano for example with
reasoning_effort: minimal
gives like 10000~15000 Ms on average and always the cost=0 does anyone have explanation for this?? I'm using the nodejs package!! and here is the provider sent "providers": [ { "id": "openai:responses:gpt-5-nano", "config": { "max_completion_tokens": 4000, "max_output_tokens": 4000, "apikey": "api-key-here", "tools": [], "tool_choice": "auto", "reasoning": {"effort": "minimal"} } } ],
I'm sending the reasoning effort as minimal, and it still uses the default medium effort, how can I override this?
u
Hey, I can take a look. Are you on the latest version?
a
Hey, thanks for replying It's solved now after updating to the latest version, I was using v0.117.1 but after investigating the latest commit on github, with v0.117.5 it works thanks a lot 🙏