Fixing bug where critic in GAIA always flags outpu...
# proj-agent
s
Fixing bug where critic in GAIA always flags outputs as false, resulting in 3x too many inference attempts https://github.com/OpenHands/benchmarks/pull/439 kudos to nvidia team for seeing this @millions-crayon-32880 @limited-student-10747
✅ 1
m
Okay, this should not impact results, but might impact costs. We could retroactively recalculate gaia's results for other models dividing in 3, or leave them as is. @limited-student-10747 thoughts?
👍 1
l
If it'll fix future runs we should fix past runs too
m