Hi everyone! Is there any particular reason why benchmarks repo (for swebench) never removes binary diffs from the agent's patch? I see this function here: https://github.com/OpenHands/benchmarks/blob/9b40b1a18e0fbd45ea0994d63cd1cc44bb2d2270/benchmarks/utils/patch_utils.py#L72 but this is never used by the eval code? I remember the legacy openhands swe bench code used to do this

thanks Aditya! I suspect it slipped, seeing if we should apply it on patchs here https://github.com/OpenHands/benchmarks/issues/654

I think this will change scores for swe bench multimodal afaik because it often ends up editing images/PDFs

@busy-branch-28834 for most benchmarks it seems to be less than <1% difference. but Gemini-3.1-pro deserves a closer look, it can be up to 2%~

Ok thanks! I asked only because I thought it is intuitive to do this since we already have the code for it. Glad that it doesn't affect numbers!