https://docs.all-hands.dev/ logo
Join Slack
Powered by
We should have this feature from Hermes to fork a ...
# proj-agent-canvas
b
We should have this feature from Hermes to fork a conversation starting from a specific message:
h
Any specifics on what you use it for @better-monitor-61654 ? I've used it a few times in claude but never found a consistent use case. I'd love to understand how you or anyone else uses it and what it helps solve. I can write up an issue for it afterwards 🙂
l
I think we might have this already but not exposed in the UI?
nod 1
c
Pretty sure 
conversation.fork()
is supported in agent-sdk
🤯 1
b
it's super useful when you are working on a specific task and you realize you need to fix another issue first (reusing the same context)
once you fixed the issue, your context is full, starts to rotten or gets compacted
so you want to resume where you left before starting to fix that issue
it's all about reusing the context
c
nod There are many times I think, “good thing GPT-5.5 is so easily steerable, so I can take a detour and tell it to come back to the main thing”, but it’s still on my mind
b
context engineering is still a thing
the condensers are not perfect
the agentic memory problem is still open
thus, having tools to manage the context is super useful
a workaround would be: /copy or /clone to create a /new conversation but copying all existing events and then just edit any of the messages in the copied conversation. This would also create a fork starting from the edited message.
BTW: /copy or /clone commands don't exist either. What I do is /new and then copy the events folder, and restart the agent-server
c
What LLM do you typically use, Christopher?
b
Qwen-3.6-27b
always local
I'm quite happy with it
I don't need more than that
and since I'm using it, the agent/harness became the bottleneck
which is all a success
c
Ohh, interesting, thank you. You probably do need more careful context management, yes. But it sounds amazing it works already… a 27B !
b
yes
it's amazing
didn't expect it so soon
c
What context window?
b
200.000 tokens, but I try to stay under 120.000
nod 1
and yes, since I'm running locally, I take care about the prompt cache
the good thing is I have total control of the prompt cache
I don't power off the server, only sleep
when you use cloud providers, your prompt cache gets deleted after some time
so speed is acceptable
especially for agentic-workflows (not so for vibe-coders)
c
Just out of curiosity with this setup, when you say locally, do you mean on your remote VM? I mean, I thought I understand from other discussions you prefer running OpenHands app-server on a remote VM. Is Qwen inference running on the same VM?
b
I have a local server running llama.cpp and OpenHands
it works 100% offline
c
Aha, awesome
b
but I can access it from the cloud also
nod 1
by creating a ssh tunnel and exposing the "one single endpoint"
m
Actually, i was looking inside a similar feature (a superset, it is more powerful) that Pi has. You can see more here: https://github.com/OpenHands/software-agent-sdk/issues/3750 My idea was that you can fork fhe conversation at every message (user or agent) to be able to reuse context. We have already a big part of the machinery involved, i.e, the fork method. What is missing is making the trajectory of a convo a tree and not linear as it is now. This is not a big change. We just need to be able to know where a given message is in a convo. So the changes to do are somewhat minimal.
b
we have event ids for every message
I'm doing the fork manually by copying and deleting .json files in the events folder
why do we need to create a tree?
you want to keep references?
maybe better seperate into two features (the fork and the tree-stuff)
c
I think the UX that a tree-like history allows is like: go on a tangent; once that is done, come back to the message; and continue normally. The branch did its thing, without the user needing to open a new tab for the other conversation and click around to come back
b
yeah, to keep a reference
but I see this as an additional feature
c
Thanks for the link, Vasco, I wasn’t aware of it, looks very good
It is a superset, yes
m
On "why the tree instead of just deleting/copying events" and silent corruption on deletion is the strongest argument for the tree. The condenser forgets events by ID and apply() silently skips any ID it can't find — so deleting events doesn't crash, it corrupts quietly: https://github.com/OpenHands/software-agent-sdk/blob/7c93fc0795573ad599a9c505d743b1a28c23eb2b/openhands-sdk/openhands/sdk/event/condenser.py#L83-L96 Concretely: • EventLog scan stops at the first index gap and drops the whole tail → delete one middle event file and you lose everything after it: https://github.com/OpenHands/software-agent-sdk/blob/7c93fc0795573ad599a9c505d743b1a28c23eb2b/openhands-sdk/openhands/sdk/conversation/event_store.py#L245-L254 • delete the Condensation event → context un-condenses and balloons back (only a Condensation triggers apply): https://github.com/OpenHands/software-agent-sdk/blob/7c93fc0795573ad599a9c505d743b1a28c23eb2b/openhands-sdk/openhands/sdk/context/view/view.py#L117-L122 • delete a non-forgotten event before summary_offset → the summary inserts at the wrong index → scrambled history to the LLM (same apply link above). The tree avoids all of this by construction: forgotten events are always a contiguous slice of the view, i.e. ancestors of the condensation → https://github.com/OpenHands/software-agent-sdk/blob/7c93fc0795573ad599a9c505d743b1a28c23eb2b/openhands-sdk/openhands/sdk/context/condenser/llm_summarizing_condenser.py#L278-L281 — and the proposed path_to_root(leaf) is a contiguous ancestor chain, so every forgotten ID is still present and summary_offset stays valid. Nothing is ever deleted; you just move leaf_event_id. tl;dr: the manual copy/delete workaround works until a condensation lands in the cut, then it fails quietly. The tree is pointer-move only → no GC, no dangling refs, condenser-safe. Fork-from-event can still ship first; the tree is what makes reusing context actually safe.
b
Does this hold true if we only delete events at the end of the context? Those deleted events are expected to not be included in the condensed context.
m
Does tail-only deletion still trip the corruption modes?
Mostly no, you're right. For a contiguous block of the newest events: no index gap (scan stops cleanly), 
summary_offset
is before the deleted tail so 
apply()
is unaffected, and un-condensing only happens if you delete back through the last Condensation. So a tail strictly newer than the last condensation is safe (uncondensed = clean truncation).
Are those tail events expected to not be in the condensed context?
No. The condenser forgets a contiguous middle slice and keeps the recent suffix. Condensed context = [head] + [summary] + [recent tail]. The end of context is the kept tail, it's sent to the LLM, so deleting it drops live history, not forgotten material. Two caveats even within tail-only: "end of context" (the View) ≠ highest store indices, non-view events (
CondensationRequest
, state updates) can sit on top and reintroduce a gap; and a tail delete can orphan a tool_use from its tool_result. I believe the cool feature would be to be able to fork at every user message, right? The changes to have this machinery are not so much. I think we could have it pretty fast.
b
Sure, but the problem is the prompt cache
I usually prefer to condense myself
But it's a complete different condensing strategy
Any way, I didn't know that the events Ids are referenced by the condenser and that messing with the events could have this effect.
I would expect that a deleted event (invalid event reference) is just left out by the condenser
Maybe the tree idea could be useful when using SGLang, which offers better prompt cache management
so that SGLang keeps a checkpoint for the point where the convo diverges (the tree branches)
at the moment, I'm only appending / removing events at the end
c
We have some mechanism in place to skip some invalid stuff like a tool result without its call
b
how does this happen?
it's when the condenser removes the tool call?
c
yes, and the other way around, tool call without result or with two results: from runtime errors
b
ok
I think this strategy of removing specific events and leaving others is very tricky and not efficient
What I describe in https://github.com/OpenHands/software-agent-sdk/issues/1496 is that the LLM is processing a big new not-cached prompt twice
for the same cost, we can generate a condensed prompt to continue the work. PP is for free (it's cached) and TG is cheaper then doing PP again on the resulting condensed prompt that breaks the prompt cache because the prompt-prefix changed (because events were removed).
super compressed...maybe too compressed and loosing details
but I think the details shuold be fetched on-demand if necessary directly from the conversation logs
but again, I think this is a completely different condensing strategy
with pros and cons
going back to original fork topic: if deleting events at the end is safe, maybe we should offer this before implementing the tree thing
c
I've been looking into history-as-tree before and I’m a fan of experimenting with it. I think it can allow us to offer easier nice UX. I’ve seen people perceiving the effects of (go on a tangent; come back) as an almost infinite context window I don’t have strong opinions on order, up to you Vasco if you feel it’s reasonable enough. I do agree with Christopher that turning to a condensing strategy that doesn’t keep tail may be good, if we can get the condenser prompt good enough so that the LLM can continue normally. I'm not sure if I’m missing something about the deletion at the end - my feeling there is that it’s risky and may involve tricky condenser changes and client-side changes too, which ups the complexity, but I could be wrong.
b
and the same applies for the "edit message" button
when you edit a message, all following messages must be deleted
or logically deleted, in case you want to keep all branched traces
like when in ChatGPT you see the number 2/2, meaning you are watching the second try
BTW, I started to delete events at the end to workaround https://github.com/OpenHands/software-agent-sdk/issues/2887
(convo corruption)
I also copy events from one convo to the other
I have never noticed any issue with the condenser
c
Isn't that fixed? 2887
or logically deleted, in case you want to keep all branched traces
Yes, logically deleted. This sounds like a fork, in the sense of branching conversation from the (edited event - 1), with history for both
b
actually, I thought the condenser just works with the events in the events folder. I didn't know there were other references to those events.
c
the condenser has a list of 
forgotten event ids
, so it knows what to replace in LLM context
do you mean something else?
b
Isn't that fixed? 2887
Probably. I will report back if I see it again.
🙌 1
the condenser has a list of 
forgotten event ids
, so it knows what to replace in LLM context
Well, in that case, we would have an invalid reference to a deleted event (a forgotten deleted event). But in case those references are just indexes (probably), then new events would be referenced/marked as forgotten. In this case, the invalid references should be deleted together with the events.
anyway, I think memory (context) should be dynamic: we should replace info in the context with summaries and links to that removed information, so nothing gets forgotten
ie. the agent would always be able to remember if the needs to
like bubbles inside bubbles
m
Personally, I would go with the tree. We can test it better w.r.t the current condenser (and errors with the condenser are generally a huge pain to debug). The backend work for the tree shouldn't take very long. In the meantime, we can decide how to present it nicely in the UI. We could also consider giving users the option to choose which condenser type to use. For instance, there are at least 4 types of condensation (though there are probably many more): • Summarization • Sliding window • Truncation • State checkpoint I think the one we are currently using is a hybrid of these.
nod 1
b
yes
This tree idea you mention, is it only meant to store a conversation with branches while avoiding redundancy, or does it go further and also consider the possibility of collapsing a group of events into a summary event (creating abstractions, like bubbles inside bubbles), with the option to have the LLM expand those collapsed event branches again when needed?
I always imagined that an LLM's context/memory should be a view of a dynamic knowledge tree that can be expanded or collapsed based on the current focus of attention.
but the nature of LLMs is append-only
if we change the context prefix all the time, this planet would burn alive
Prompt cache is all you need :-)
If we change the prefix, the cost for every message increases from 617 to (78.135 + 617) => every message costs 130 times more
Imagine when we have a context of 200.000 tokens
m
This tree idea you mention, is it only meant to store a conversation with branches while avoiding redundancy, or does it go further and also consider the possibility of collapsing a group of events into a summary event (creating abstractions, like bubbles inside bubbles), with the option to have the LLM expand those collapsed event branches again when needed?
We can do that.
if we change the context prefix all the time, this planet would burn alive
Ideally, the branches will use the prefix-prompt up to where they appear.
c
Just to note, for the condensation mechanism. The alternative that maybe we could try, to see how it can work, is like: • summarize all events ◦ may want to keep user intent • next call is (system, user message, summary) Currently it is • summarize old 1/2 events • next call is (system, user message, summary, recent 1/2 events) Claude Code was doing something like • summarize all events ◦ summary contains each turn’s (user message, summary of agent turn) • next call is probably (system, summary) I think Codex is/was doing something like • summarize all events ◦ summary tells it it’s just restarting and it should investigate first ◦ not sure if it contains user messages, but I think so • next call is (system, summary) Hmm, while drafting this I realize I know much less about Codex than I should…
đź‘Ť 1
b
Claude Code and Codex probably append the condensation instruction reusing all prompt-cache
I'm doing it this way to save tokens and to control the context
Generate a condensed prompt
Copy code
#### General instructions for generating condensed prompts

When asked to generate a condensed prompt, stop all tasks and do NOT produce any report or answer other than what is requested below. Ignore all previous tasks except this one.

Generate a **condensed context prompt** (a highly detailed but compact summary) of the entire previous conversation.
This prompt will be given verbatim to you as **your only context** to continue and finish your work.

### MUST INCLUDE (only if present in the conversation):
- The **requirements and goals** defined by the user, using same words and detail.
- All **pending tasks, open questions, or incomplete objectives**
- The **last uncompleted user instructions** before asked to generate a condensed prompt.
- All **non-obvious information** explicitly provided by the user
  - credentials, commands, tokens, passwords, configuration values
  - explain what each one is used for if it is not self-evident
- All **important discoveries, conclusions, constraints, or decisions** reached during the conversation
- Any **mistakes, dead ends, or incorrect assumptions** that were identified, and the lessons learned to avoid repeating them
- Any **non-obvious acronyms, abbreviations, or domain-specific terms**, including their meaning
- All required readings (ixai-instructions / read-markdown). Example: "Read @example1.md, @example2.md, @example3.md (read-markdown)"

### MUST NOT INCLUDE:
- Obvious or generic information
  - anything that can be trivially inferred by common knowledge or by reading the condensed prompt itself
- Redundant explanations
- Unrequired info about completed tasks
- Meta commentary about how the summary was created
- Any content that was not explicitly present or clearly implied in the conversation
- Info that will be invalidated after restarting the server (state of running processes)
- Info about changes that can better be obtained executing "git diff" (but include the instruction to perform a "git diff" to see changes in progress)

### STYLE & FORMAT REQUIREMENTS:
- Output **Markdown only**
- Be precise, structured, and information-dense
- Prefer bullet points, sections, and short paragraphs
- Do NOT add new ideas, guesses, or improvements
- Do NOT omit details for brevity if they are required to continue the work correctly
- All passwords must be placed in one single section at the beginning

**Goal:**
A different LLM must be able to read this condensed prompt and continue seamlessly, without asking follow-up questions or repeating previous errors.
copy/paste this to a conversation and you get a condensed prompt that you can copy/paste into a /new session
an improved version would add references to a more detailed summary or the detailed events
or real-time generated summaries guided towards some specific question or requirement
which should be cached and listed for fetching on-demand
m
2 Views
Open in Slack
PreviousNext
Powered by