When running the swe benchmark inference, some of my containers hang because of rate limiting. I get the error message:

Instance pylint-dev__pylint-4604 - 2026-06-24 00:45:46,851 - WARNING - REST has reported terminal status finished for 31s with no post-run WebSocket snapshot; accepting as final to avoid an indefinite hang (elapsed: 1006.5s). This may indicate a WebSocket delivery issue.

The logs also show that nothing was ever done in the container since the LLM calls were never successful. This is fine for me. However, when running the SWE bench eval I generate all the patches for the containers in

results.swebench.jsonl

. This file shows that a patch was generated from these "dead" containers. Here is an example:

{"instance_id": "sympy__sympy-23534", "model_patch": "diff --git a/sympy/core/symbol.py b/sympy/core/symbol.py\nindex f4a555c13f..2aca8bab27 100644\n--- a/sympy/core/symbol.py\n+++ b/sympy/core/symbol.py\n@@ -791,7 +791,7 @@ def literal(s):\n        return tuple(result)\n    else:\n        for name in names:\n-           result.append(symbols(name, **args))\n+           result**ppend**ymbols(name, cls=**s, **args))\n \n        return t**e(names)(result)\n \n", "model_na**_or_path** "OpenHands"}

This h*owever makes no sense to me as the* container didn't do anything (Si*nce the LLM was never successfully* called). Does anyone know why there is still a patch?

Hmm, that’s a good question… The diff is still captured when the agent fails midway through, but I’m not sure why there would be a non-empty diff. If you have a way to reproduce we could dig in further

Here are the upper logs:

2026-06-23 17:30:02,450 - INFO - openhands.workspace.docker.workspace - Docker workspace is ready at <http://127.0.0.1:36474>
2026-06-23 17:30:03,398 - INFO - openhands.sdk.skills.skill - Found 50 skill files in public skills repository
2026-06-23 17:30:03,419 - INFO - openhands.sdk.skills.skill - Loaded 50 public skills
2026-06-23 17:30:03,633 - INFO - benchmarks.swebench.run_infer - repo_path: /workspace/sympy/
2026-06-23 17:30:04,836 - INFO - openhands.sdk.conversation.impl.remote_conversation - run() triggered successfully: <Response [200 OK]>
2026-06-23 17:30:35,884 - ERROR - openhands.sdk.conversation.impl.remote_conversation - Request failed: timed out
2026-06-23 17:30:35,885 - WARNING - openhands.sdk.conversation.impl.remote_conversation - Error polling status (will retry): timed out
2026-06-23 17:33:11,676 - WARNING - openhands.sdk.conversation.impl.remote_conversation - REST has reported terminal status finished for 3 polls without a post-run WebSocket snapshot; continuing to wait for the authoritative snapshot (elapsed: 185.8s)
2026-06-23 17:33:40,115 - WARNING - openhands.sdk.conversation.impl.remote_conversation - REST has reported terminal status finished for 30s with no post-run WebSocket snapshot; accepting as final to avoid an indefinite hang (elapsed: 214.3s). This may indicate a WebSocket delivery issue.
2026-06-23 17:33:40,182 - INFO - benchmarks.utils.fake_user_response - Sending fake user response #1: Please continue working on the task on whatever ap...
2026-06-23 17:33:40,230 - INFO - openhands.sdk.conversation.impl.remote_conversation - run() triggered successfully: <Response [200 OK]>
2026-06-23 17:34:56,364 - WARNING - openhands.sdk.conversation.impl.remote_conversation - REST has reported terminal status finished for 3 polls without a post-run WebSocket snapshot; continuing to wait for the authoritative snapshot (elapsed: 75.1s)
2026-06-23 17:35:24,798 - WARNING - openhands.sdk.conversation.impl.remote_conversation - REST has reported terminal status finished for 30s with no post-run WebSocket snapshot; accepting as final to avoid an indefinite hang (elapsed: 103.6s). This may indicate a WebSocket delivery issue.
2026-06-23 17:35:24,866 - INFO - benchmarks.utils.fake_user_response - Agent finished with FinishAction after 1 fake responses
2026-06-23 17:35:24,867 - INFO - benchmarks.utils.fake_user_response - Conversation completed. Total fake responses sent: 1
2026-06-23 17:35:25,327 - INFO - benchmarks.swebench.run_infer - [INSTANCE] [32mOK[0m patch=[32mNONEMPTY[0m commit=[32m0[0m changes=N msgs(a/u)=1/2 tool_calls=[37m45[0m errors(agent/conv)=[32m0/0[0m end=[32mfinish_tool[0m [2mpreview='diff --git a/sympy/core/symbol.py b/sympy/core/symbol.py\nindex f4a555c13f..2aca8bab27 100644\n--- a/sympy/core/symbol.py\n+++ b/sympy/core/symbol.py\n@@ -791,7 +791,7 @@ def liter…'[0m
2026-06-23 17:35:25,328 - INFO - benchmarks.utils.evaluation - [worker] done id=sympy__sympy-23534
2026-06-23 17:35:25,460 - INFO - benchmarks.utils.evaluation - [worker] Saved conversation archive for sympy__sympy-23534 to eval_outputs/princeton-nlp__SWE-bench_Verified-test/openai/qwen-3.6-27b_sdk_c950fdb_maxiter_500/conversations/sympy__sympy-23534.tar.gz
2026-06-23 17:35:27,460 - INFO - openhands.workspace.docker.workspace - Stopping container: ac417781dd6fcb63c73c3aae392b075bb99f893cd188064049419165953c3f82
2026-06-23 17:35:27,461 - INFO - openhands.sdk.utils.command - $ docker stop ac417781dd6fcb63c73c3aae392b075bb99f893cd188064049419165953c3f82

Maybe the patch is from a previous run? Maybe the container wasn't actual clean for this run?

Thanks, is there a particular command you can run to cause this error to occur?

run swebench-infer .llm_config/config.json \ --dataset princeton-nlp/SWE-bench_Verified \ --split test \ --num-workers 1 \ --max-iterations 500 \ --n-limit 100 \ --workspace docker

This is on the benchmarks repo right?