When using GPT5 as a judge i get this very often. I don't get why. My intuition is that it's happening when the answer to be judged is pretty long and maybe there is a timeout in some level of the llm-call. Could this make sense? Should i set some timeout config in the provider to be sure? https://cdn.discordapp.com/attachments/1461732259353595916/1461732259525693703/image.png?ex=697cc340&is=697b71c0&hm=3f7ebcc13aa0c0e9d4491c7fede32c15e0fc04dc52c467256a369dad90aa1d10&

Can you either share a config to help me reproduce and/or can you re-run your eval with --verbose and copy paste (or email me - michael@promptfoo.dev) the relevant section of the logs? I am guessing that you are running into a max token limits with reasoning. We will be able tell for sure with the logs.

i will try and reach back, thank you 🙂