https://docs.all-hands.dev/ logo
Join Slack
Powered by
Every Docker conversation kept its package caches,...
# proj-agent
m
Every Docker conversation kept its package caches, installed dependencies (node_modules, .venv) and crash dumps forever. We are trying to solve this problem here. What we did, as 4 stacked SDK PRs: • github.com/OpenHands/software-agent-sdk/pull/5263|#…: new storage module; drops a conversation's caches when its container stops, and disables core dumps • github.com/OpenHands/software-agent-sdk/pull/5407|#…: opt-in disk budget; above X% full, stopped conversations shed their git-ignored dependencies, oldest first • github.com/OpenHands/software-agent-sdk/pull/5429|#…: opt-in retention; after N inactive days the sandbox is deleted, the history stays readable • github.com/OpenHands/software-agent-sdk/pull/5433|#…: local mode; deleting a conversation now also removes its git worktree (the branch is kept) Reviews/feedbacks welcome 🙏
🙌 2
👀 1
q
Taking a look!
High level IIUC the storage and deletion abstraction is only meant for files the agent server can touch directly So this won't extend for remote runtimes. As long as the remote runtime has an agent server running it will handle deletion accordingly?
m
I understand your point, but I don't know how to answer 🙂 I think we do have the high level interface and then you can add adapter for your runtime type. In this case we do have just local and docker runtime. And both are actually local runtimes. So the naming is not so good
q
Yeah I think the PRs also heavily assume path based cleanup everywhere so its not clear whether the abstraction as it is today will easily extend for remote use case
e.g these configurations
Copy code
root: Path
StoredRuntime.home: Path | None
Trash.discard(path: Path)
path.rename(...)
safe_rmtree(...)
shutil.disk_usage(...)
m
Oh yes I agree with you... it is more to control local runtimes... Should we also do for remote ones? or ?
q
I think if we assume a remote runtime runs an agent server, then local only assumption works and we can avoid over engineering ? Just wanted to confirm whether thats a reasonable assumption
i've approved the first PR for now, and left some feedback too LMK what you think
m
mmmh i think for remote runtimes can be a problem right? Because in a remote runtime we will have the agent-server managing the sandbox where it lives right?
So yes, we could have such a solution just in the local case... but then how are we managing the remote one?
q
Right so the agent server inside the remote sandbox would reclaim space on sandbox disk? I think the case where we run an agent conversation on a remote sandbox, without an agent server inside the sandbox, would be a problem; but idk if that's ever the case in practice; as in can we assume any sandbox type should package an agent server inside it
I think APIs to remotely control the reclaim strategy on the agent server might be something to consider, unless im misunderstanding something here
According to openhands Case A: remote agent-server manages its own local sandbox
Copy code
client / orchestrator
  -> remote agent-server
      -> local Docker/container/workspace/storage
Current abstraction works. Case B: central agent-server manages remote execution service
Copy code
central agent-server
  -> remote sandbox API
      -> storage inaccessible as local Path
I think we mostly operate on case A, do you think it would be easy to defer case B support to future PRs?
m
I have to check but we could have the abstraction to menage both cases or?
👍 1
q
Cool if its not terribly difficult
m
perhaps we could do something in this direction
Copy code
class RuntimeBackend(Protocol):
    def runtimes(self) -> list[RuntimeRecord]: ...       # id, last_active, state (running/paused/stopped)
    def idle(self, id) -> AbstractAsyncContextManager[bool]: ...
    def supports(self, step: Step) -> bool: ...           # local worktrees: no RETIRE; remote: maybe no DEPENDENCIES
    async def apply(self, id, step: Step) -> bool: ...    # do the step; True if something was freed
    def usage(self) -> float | None: ...                  # local: disk fraction at root; remote: sandbox count / quota

class FilesystemBackend(RuntimeBackend):   # shared base: Trash + selectors
    ...                                    # Docker, local worktrees, Podman, Apptainer subclass this
class CloudSandboxBackend(RuntimeBackend): # apply() calls pause() / DELETE
    ...
But the most important question is: do we want that? or?
q
hmm i suppose the main difficulty stems from the fact that the semantics are different between local and cloud a stopped sandbox probably means its no longer accessible vs for local docker convos stopped container is the signal to reclaim space
m
Could you give me a couple of examples of that? I'm not so an expert in that 🙂
I explored a little this. See attachment for more info. cc @quick-window-91705
runtime-reclaimer.html
btw, the solution seems to work. The VM is still up 🙂
@calm-artist-7206 interested in your opinion
Perhaps is overkilling and we should just look at the local case
c
Just to note, from the HTML doc - this isn’t accurate, is it? The agent-server can manage its space on its box.
I'm reminded that an essential principle of agent-sdk/agent-server when we’ve designed it, was that from the perspective of the agent-server, everything is local to it. So the same thinking that applies to fully local applies to “local inside a box”… if that makes sense
Re: PR 5429 - there is a concept of “archived” conversation on the Cloud, which means read-only: in practice, the user can open the conversation history and look at it, but cannot prompt anymore. It sounds like a similar thing?
Re: 5263, just a side note about mocks in tests I think I’m starting to be wary of ALL mocks in tests. Those tests could be a real-life test, on an actual agent-server, with two docker inner conversations and the rules we’d like to enforce. That would actually show us, and our agents, that the real thing does work. IMHO what happens is that our old assumptions that unit tests are “easy and normal” while realistic tests are “hard and time-consuming” have flipped. They have flipped because of agents getting good. Agents can write infinite mock-based “tests” that don’t guarantee anything and humans don’t read anymore so no one knows, while, on the other hand, they’re actually able to make the entire scenario work. The entire scenario, the real-like thing, is not anymore a “mkay maybe I’ll try once before merging a risky PR”, it becomes a “set up a real-like scenario, put it in markdown, let the agent do it every now and then. Then you know the thing works.” My poor brain tries to make sense of it…
Re: retention days. Left a tiny comment as food for thought, it’s not really a problem: https://github.com/OpenHands/software-agent-sdk/pull/5429/changes#r4167132791
Perhaps is overkilling and we should just look at the local case
Yes, I think we don’t have a case of a sandbox without agent-server. In fact, and please correct me if wrong, I don’t think the new “inner” agent-servers can be in something else than Docker?
Rohit, do you have something in particular in mind, that would be a remote runtime without at least one agent-server?
We're stepping again a bit on the V0 path with the “inner” docker runtimes and the term “runtime”, but I don’t think we got to remote runtimes that sit there waiting to do stuff without to an agent-server
(as it used to be)
q
i think its probably safe to assume an agent server is running on the remote machine; if not then the user would be reinventing agent server to remotely control conversations anyway i was mostly curious whether we want to support reclaiming remotely when users roll out their own lightweight bespoke agent server; i think we shouldn't
nod 1
c
TBH somehow I have the feeling, in the stacked PRs, that maybe the reclaimer could be an external script, external to the agent-server. It feels a bit like a background task that would be run as maintenance It's not clear to me though, if that’s easier or harder now 😅
m
Hey 👋 worked a little bit on that. So taking inspiration from your reviews and comments and from our runtime-api for the cloud solution plus the new DigitalOcean one, I put together an architecture that could acomodate our needs. (see HTML attached) The principles are: • Everything is local to the agent-server. • Whoever created the box owns its lifecycle. • The remote extension point is HTTP, not a Python protocol. (Orchestrators already reach the agent-server over HTTP. A few endpoints serve every future provider; a 
RuntimeBackend
hierarchy in the SDK would serve none of them better.) • Native first. When the provider has idle and expiry settings, set them at creation and let the provider enforce them. Only build a loop for what it can't do. • Timers need an always-on owner. I would also like to have the opinion of @fierce-account-55183 as you are an expert in runtime-apis 😉 Please let me know what do you think. I hope i didn't forget nothing important
runtime-reclaimer.html
cc @calm-artist-7206 @quick-window-91705 @limited-student-10747
cc @rhythmic-sunset-66788 @dry-book-58714
Cc @curved-angle-80449
c
@bland-bear-604 lets take a look at the last attached runtime-reclaimer.html , can you pick it up from Slack? Remember to identify users correctly, and it’s from Vasco.
m
🫣
😹 1
c
(my Mac’s disk got full… like, really full, that’s why the poor thing got cut off while it was reading)
m
haha ok don't worry 🙂
So continuing on that. I opened a couple of issues to track the progress: • [Feature]: Docker runtime: reclaim per-conversation caches so runtime-data stops filling the host disk • [Feature]: Land the conversation-storage PR stack (#5263 → #5433) on current main • [Feature]: Agent-server storage API: status, reclaim-now, per-conversation retention I'm starting to merge the Prs in the stack above. Still need a review for the last one 🙂 @calm-artist-7206 if you want/if it is better for you I can split the PR
cc @quick-window-91705
About your comment @calm-artist-7206 github.com/OpenHands/software-agent-sdk/pull/5429#…
You would like something more dynamic right? Such that you can decide per conversation/automations what is the retention? It make sense to me. Perhaps we should explore that more
c
Ah so glad we were talking about this. I had just started on a PR somewhere that would allow one to extend the lease on a cloud sandbox or specify longer lease at creation time. We have a couple of customers that need that. Not sure my approach is the best one... Let me see if I can find that.
c
FWIW, my agent did figure something out, but it didn't get to post. Not sure, you may want to feed your agent this (or not, seems to equivocate a bit as LLMs love to do):
Copy code
smolpaws: I like this revision. The ownership split is right: the agent-server cleans files it understands; the service that created the sandbox manages its lifetime. That keeps provider-specific code out of the SDK.

I'd tighten four things before calling it ready:

•⁠  ⁠Export must succeed before deletion. "Downloaded" should mean the archive reached durable storage and was checked—not just that the server started sending it. History export also does not necessarily preserve uncommitted work. The confirmation should say exactly what survives.

•⁠  ⁠Old does not mean safe to delete. State-file timestamps can help choose candidates, but the reclaimer must check again under the idle lock before removing anything. Shared caches and dependencies need particular care: another conversation may still use them.

•⁠  ⁠Dry run is a preview, not a promise. Work can restart between preview and apply. The apply pass must recheck and skip anything that became active.

•⁠  ⁠"Archived" currently promises more than export delivers. The document says history stays readable, but also says importing exported history is a follow-up. A downloaded zip and history readable in Canvas are different user experiences. I'd make that gap explicit.

The warning about polling is especially good. Keeping the agent-server's idle clock unchanged is insufficient if opening its tunnel wakes the provider. Native timers avoid that trap.

My take: keep this architecture, and spend the next effort on safe deletion and recovery—not a broader provider abstraction. The proposed storage-status and reclaim endpoints seem useful even with one provider.
FWIW my main thought seems solved, I do think the client that created the box (or a maintenance script of sorts) should have responsibility
m
Ah thanks. We can always fix them ;)
Nothing is released yet
c
Found it. github.com/OpenHands/runtime-api/issues/758 My plan was to add a 
lease_expires_at
field to the Runtime model in runtime-api and a 
POST /extend_lease
endpoint that lets an owner tick the lease, preventing idle-reap regardless of idle time (still capped by the 12h session max). I would then add a companion 
POST /sandboxes/{sandbox_id}/extend-lease
in OpenHands/enterprise that passes through to runtime-api. This lets warm-pool controllers hold pre-initialized sandboxes and late bind to conversations (which some customers need due to extensive setup some of which can't be baked into a custom image). I admit the term lease is ambiguous here and should be changed in my proposal since we have three kinds of leases in cloud: 1. session-idle (my proposal), 2. cleanup of pods after snapshot (seems like this is internal only and shouldn't be exposed in user api), 3. max sandbox life time (this could be managed via an api too). I see your rev 3 design prefers setting per-sandbox 
idle_timeout_seconds
once at creation and has no extension mechanism. Should I realign on that or could we support both?
Open in Slack
PreviousNext
Powered by