I’m enjoying the openhands index and I’m already using it to inform deployment decisions for coding agents. GLM-5 seems to be a strong open source contender but we are lacking the overall picture because the frontend results (SWE-bench multimodal) seem to be missing, right? Are there any plans to run that suite to complete the picture for GLM-5, please?

Hey @important-diamond-63626, we’re working on it! We’re actually far behind our final goal of benchmarking the most important new models in ~2 days due to some infrastructure issues we’re having. Running these evals smoothly is difficult for a number of architectural reasons, and we’re still working through them (and at the same time, the rapidly evolving SDK is a moving target). But hopefully we’ll be done soon! @millions-crayon-32880 and @strong-ice-84145 are working hard on it

Hey @important-diamond-63626 we just updated the OpenHands Index with the GLM-5 SWE-bench Multimodal result! Please see the results here https://index.openhands.dev/home

Wonderful - and somewhat surprising 🙂 Next I would love to see Gemini 3.1 Flash Lite and Nemotron Super. Any idea when these might be added?

Hey @echoing-garage-24541! Nemotron Super - we are evaluating it! Qwen3-Coder-Next - we are evaluating it! Qwen3.5-397B-A17B we are not evaluating at the moment, but we are evaluating Qwen3.5-flash Gemini 3.1 Flash Lite we are not evaluating at the moment.

Amazing @millions-crayon-32880, that puts GLM-5 at the top of open source models but only ever so slightly over Kimi-K2.5, which is one order of magnitude cheaper to serve at scale. Fascinating! Thanks for helping me to complete the picture 🚀