I'd love to have the ability to instantly stop the agent rather than having to waiting up to minutes in the worst case 1. https://github.com/OpenHands/software-agent-sdk/pull/2206 This one I've had working for a while, but it was hacky. It's using async llm completions underneath while masquerading a sync api interface (but less surface area) 2. https://github.com/OpenHands/software-agent-sdk/pull/3284 This one introduced async everywhere as an option in addition to sync. But larger surface area to maintain Still cleaning things up, but each come with their pros and cons any feedback on preferred direction?

CC @straight-sundown-84319

cc @calm-artist-7206 as well

I really like the async / sync solution! (3284) • async paths • sync paths As the PR shows, litellm is doing the same (

acompletion

,

aresponses

etc), so we plug in our stuff quite nicely on their async execution paths. It’s more code to go through, but it feels to me like the execution is actually cleaner and with less landmines. 🤔 And being pattern-y, I feel like the agent will keep it straight easier maybe

I read through both PRs. Here's my comparison: PR #2206 — async internally, sync API surface • Runs

litellm.acompletion()

inside a hidden daemon thread with its own event loop • Exposes

conversation.interrupt()

that cancels the async task • Sync API is unchanged — callers don't see async at all • ~1750 lines added PR #3284 — proper async/sync dual paths • Adds

a

-prefixed async methods at every layer:

LLM.acompletion()

→

agent.astep()

→

conversation.arun()

• Plugs directly into litellm's native async APIs (

acompletion

,

aresponses

) • Agent server prefers

arun()

when available, falls back to thread-pool for backward compat •

CancellationToken

for cooperative cancellation across the stack • ~2350 lines added, sync paths completely untouched My take: #3284 is the right approach. The hidden event loop in #2206 is exactly the pattern that causes async nightmares — a secret background loop masquerading as sync. When something goes wrong (and it will), debugging is much harder because the async machinery is invisible. This is the kind of thing Trio was designed to prevent. #3284 follows the established Python convention: every sync method gets an explicit

a

-prefixed async counterpart. It's more code, but it's patterned code — once you understand one layer, all 8 layers work the same way. Agents and LLMs have no trouble maintaining this. Specific wins of #3284: • No thread-pool workers tied up during LLM I/O — the event loop is free • Cancellation is cooperative and explicit (

CancellationToken

), not based on

asyncio.Task.cancel()

which can leave resources in weird states • Matches litellm's own design (`completion`/`acompletion`, `responses`/`aresponses`) • Custom subclasses get sync-delegating defaults — nothing breaks The main concern with #3284 is maintenance surface area, but since it's patterned and each async method mirrors its sync counterpart, the actual cognitive overhead is low. (This also lines up with what Engel said in the #2206 review: "I always thought that when we do this, we really should make an

acompletion

ourselves.") 🐾

Seem like 3284 is the way to go, cc @fierce-account-55183 to have a chance to weigh in as well

As long as the agent team is happy with running their code async, it works for me. Up until this point, we had already run the agent-server async and had it as the barrier with the sdk.

cc @chilly-france-44439 for visibility, if agent team gives blessings for async equivalents I can start polishing the PR

i think in the early day of the SDK, we (@wooden-umbrella-84098 ) decided to do sync only in favor of simplicity... but maybe it is the time to break that assumption since this is a real use case. I wonder if it is still possible to still maintain one source-of-truth while adding async, eg you can put implementations in the async function, and then call it from sync function -- irrc @straight-sundown-84319 had a few utils that's doing this. Would this add a lot of complexity to our system / hurt the reliability stuff (w.r.t async maintainability)

yeah I think we assumed we'd eventually support both sync and async

I think the problems we had in V0 were mostly because of making pockets of async in attempted-sync. Because it hides complexity in a bunch of special cases

I had some code doing this - but it's complicated. See • remote_workspace_mixin.py • async_remote_workspace.py • base.py

I really do think the agent can handle it better if we separate cleanly. We can adjust the prompt so it knows better. It already knows this way (the separate async and sync paths) because we definitely are not the first

Many of the problems in V0 came from enterprise - before we moved to the nested architecture

we should also tell the code review bot to look for async problems

I see your point there, they're not so many now (though... idk, we're going towards multiple agents in a long-running agent-server 🤔 ). The concept embodied here seems similar to me with something that was difficult to debug in V0: • async agent-server (edge) • async LLM (core) • plus other async code here and there We could: • tie them by attempting to "hide" async • tie them by creating clear, separate paths (a-, and non-a) The first looks riskier and error-prone to me, than the second. Sorry, I do think it seems simpler when things go well... which, well, they don't have that habit. 😅 I don't know, projects have chosen async / sync pathways through the code even in "The Before Times". Now the agent can handle the verbosity. 🤷

tie them by creating clear, separate paths (a-, and non-a)

If we can try to encourage the creation of utils function, i do think we can live with those verbosity (with the code review bot paying attention to everyline, of course)

Do folks mind with this PR as is then? https://github.com/OpenHands/software-agent-sdk/pull/3284 We can perform a followup by extracting shared utils wherever possible

FWIW, you've got my support!

@chilly-france-44439 is there a way to run a small eval set to make sure we haven't broken anything?

i've tagged

run-eval-50

which should start a small eval run!

ship it parrot but if this starts causing headaches, let's kill it and come back to the drawing board 😄

Eval seems to have ran successfully with unresolved issues comparable to prior runs I do see one failures with error

1    2.0%  Failed to process archive: not a gzip file

I'm assuming this has to do with the benchmark processing?

40/50 with claude-sonnet-4-5 is good