I'm excited for the rollout of the verifiers stack...
# success-storiese
echoing-garage-24541
03/13/2026, 9:02 PMI'm excited for the rollout of the verifiers stack!
I recently discussed the "death of SWE-Bench Verified" (due to contamination) in a segment of our AI-First Show:
One of my students commented with a couple of questions, one of them about agent benchmarking, and I just replied referencing the OpenHands trajectory-level verifier.
Aside from ensuring coding agent reliability and increasing trust, I think this could also be an important verification component in agent benchmarking.
🙌 3
l
limited-student-10747
03/13/2026, 9:05 PMAwesome, thanks for the shout out!
limited-student-10747
03/13/2026, 9:09 PMSmall correction, I think that Scale AI created SWE-Bench Pro: https://labs.scale.com/leaderboard/swe_bench_pro_public
e
echoing-garage-24541
03/13/2026, 9:10 PMOh! I was convinced it was OpenAI - thanks for the correction 😅
I guess I got confused due to OpenAI's support and evangelism there.
l
limited-student-10747
03/13/2026, 9:10 PMGreat analysis though!
limited-student-10747
03/13/2026, 9:10 PMWe’re thinking of replacing SWE-Bench Verified with SWE-Bench Pro for the next version of the OpenHands Index
👍🏼 1
e
echoing-garage-24541
03/13/2026, 9:12 PMHave you seen this? They just released SWE-Atlas
https://scale.com/blog/swe-atlas
l
limited-student-10747
03/13/2026, 9:13 PMYeah! I think we discussed it in the #C09MTM3QPLG project
✔️ 1
limited-student-10747
03/13/2026, 9:13 PMBecause of the Codebase QnA component, which we were trying to develop there