I noticed a couple of hints worth looking into: 1. It looks like “think max” reasoning mode was not activated, and the runs used “think high” instead. The HF docs say that “think high” is for “conscious logical analysis” whereas “think max” is to “push reasoning to its fullest extent”. This is based on an extracted

base_state.llm.json

from the runs with

"reasoning_effort": "high"

, though I’m not sure what this maps to in the inference provider 2. The GAIA benchmark dropped significantly from DeepSeek-v3.2-Reasoner to DeepSeek-v4-Pro (50.3 to 12.7) and a deeper look reveals that the answers are fine but are being returned in a way that trips the eval harness, which make the model appear much worse than it is in practice. Here is a small writeup about the format mismatch. What do you think, @limited-student-10747?