OpenHands

<https://x.com/claudeai/status/2032509548297343196?s=61|https://x.com/claudeai/status/2032509548297343196?s=61>

Do you feel this changes anything w/ sonnet 4.6 <@U06PYTVAES1>

e.g. Is it worth leaving behind the accuracy of 4.5 for a larger context window :thinky-spin: ?

Just thinking if this would change the index <https://index.openhands.dev/home> at all

I don't know; those evaluations were without any condensation... :thinky-spin:
I think it's possible that all those instances didn't reach 200k? cc <@U080DD1FR4K>

MRCR benchmark is not part of the index, and it's generally not a "coding benchmark"... I seem to remember that a part of the tasks were useful for coding though, in the original paper

I'd want to double-check the context sizes with <@U08H0CRU1V4>. I know there was no condensation on the index, but I'm not sure how often models ran into context window limitations or what the max size was.

back when we first tested condensation, when it happened, it did so close to the ending of the instance evaluation.

a lot has changed since then, we reduced the number of condensations and improved how we handle context, but we also are testing more languages with longer trajectories.

we are planning on doing some condensation tests soon