@brash-engine-38895 @limited-student-10747
I’m looking into this as well.
Here’s what I found so far:
- the scores for commit0 (in the Greenfield category) were recomputed with a new version of the agent (from v0.38.0 to v1.11.0) and the score dropped from 50.0 to 18.8, which drags down the overall score as well, from 52.7 to 46.5.
- it looks like the original score was done against a pre-release version of the model, code-named jade-spark-2862, as opposed to MiniMax2.5, could it have been a different model version under the hood?
https://github.com/OpenHands/openhands-index-results/commit/d5a5e9b48753d06aed0046f33060b005d14d8fb9