We ran a pretty comprehensive Eval on 12 models to...
# generalw
worried-salesclerk-85726
03/12/2026, 12:39 AMWe ran a pretty comprehensive Eval on 12 models to help us decide which one to use as our condenser model. Figured this would be valuable for others https://openhands-ai.slack.com/files/U0AHBJQJEBX/F0AL2GEE9BL/condenserreport-multimodel_20260311.md
🙌 4
worried-salesclerk-85726
03/12/2026, 12:41 AMtl;dr Qwen3-Coder-Flash is definitely good enough, but we're willing to pay a bit more for Qwen3-Coder-Plus to achieve 20% better compression.
Deepseek also very very good
w
wide-doctor-37830
03/12/2026, 1:15 AMOh cool! Can you elaborate a bit on the metrics? I can guess by the names but I'm interested in how they were computed
w
worried-salesclerk-85726
03/23/2026, 1:41 PM@wide-doctor-37830 Sorry for no reply on this, still having our dev organize all of our evals properly to get the methedologies and prompts in one place.
🙌 1
2 Views