Hey @busy-branch-28834, did you try this apptainer benchmark support? Add benchmark-side Apptainer workspace support by neubig · Pull Request #509 · OpenHands/benchmarks I haven't tried it yet but if you tried it and it worked i'll merge it in

Hi Graham sir! Sorry for the late reply - I haven't tried this PR yet but I made a somewhat similar PR for swe bench you merged that I have tried and works. At a high-level your PR seems to be sufficient for this support, basically we need two things for this to work which your PR has: 1. Add apptainer support to

prepare_workspace()

for each run_infer.py in each benchmark. 2. Change args parser to support apptainer as a cli argument. I can try your PR if you suggest - I am not fully aware of how each benchmark is configured internally (in terms of building the docker image) in the repo.

No, it's fine if we have a similar one, i'll close this!

Just to clarify, the PR I made for just for swe-bench and yours is for all benchmarks so it is a strict super-set of what I did. My PR is anyways merged

ok, maybe i'll close it for now though

Ok sir!