Sorry for being a broken record on this one but it...
# proj-agent
w
Sorry for being a broken record on this one but it's back šŸ˜… This was when the agent-server was shut down mid-conversation. So a bit of an edge case at least. But I thought we were more robust to this error happening This was while using the main branch yesterday
c
cc: @wide-doctor-37830 I think maybe we intend to apply a ā€˜hard’ condensation when the LLM API errors of this kind happen?
w
Unfortunately this error message is a symptom of a bunch of unrelated bugs. I'm on the road until tomorrow, will dig in when I get back. @wooden-umbrella-84098 can you share the trajectory for that convo? We should be handling those shutdowns gracefully but looks like some events are sneaking through. @calm-artist-7206 We don't catch these errors from the LLM like we do context window exceptions. I don't think it'll be difficult to add.
c
@nutritious-balloon-65098 Let's add these errors to the method catching context window exceptions. Open a PR.
n
I'm on it! can track my progress here.
Summary of my work since the last summary: - I confirmed the change does not force an immediate hard reset every time. - The new behavior is: - the Anthropic malformed
tool_use
/
tool_result
error is classified into the same recovery path as context-window exceptions, -
Agent.step()
then emits a
CondensationRequest
, - and the configured condenser decides what happens next. - For
LLMSummarizingCondenser
, that means: - it is treated as a hard condensation trigger, - but a hard context reset only happens if normal condensation cannot be produced. Checklist: - [x] Answered your follow-up question directly. - [x] Clarified the exact control flow distinction between ā€œhard triggerā€ and ā€œhard reset.ā€ - [x] No new code changes were made after that clarification. - [x] No extraneous changes were introduced in this step. So the concise answer is: yes, it kicks the error into the hard-condensation recovery pipeline, but no, it does not guarantee an immediate hard reset in every case.
Since my last summary, no additional code changes were made. Final status: - [x] The request has been fully addressed. - [x] The PR description was updated to make the behavior explicit: - these Anthropic malformed-history errors are intended to be captured and retried with condensed history - this does not mean an immediate hard reset every time - regular condensation may happen first - hard condensation/reset is only a fallback if the configured condenser needs it - [x] The branch was pushed and the PR was opened: #2613 - [x] The instructions were followed faithfully, including making the PR explanation explicit about original intent and the distinction between regular vs hard condensation. Conciseness: - [x] The changes are concise. - [x] No extraneous follow-up changes were added after that update. - [x] The work remains limited to the targeted classifier/test updates plus the clarified PR description.
w
Sounds like a fix, we can't guarantee a hard condensation if there's no condenser so that tracks. I'm a little worried that we have some weird race conditions or error handling that are putting the event stream in a bad place and that this will make it harder to identify them.
c
Yes, there might be at least an infinite attempt, maybe... Checking. Taking a small step back, I think maybe we could always do "hard condensation" in this case, and then we know: • system prompt • (likely) initial user message • summary • nothing else visible to the LLM, ready for the new stuff to happen
n
Final summary of my work: - [x] The original request is addressed: the PR is open and the malformed Anthropic `tool_use`/`tool_result` errors are now routed into the same recovery path as context-window exceptions. - [x] I also updated the PR wording to make the behavior explicit: this change is intended to capture these errors and retry with condensed history, which may mean regular condensation first and hard condensation only as a fallback. - [x] No new code changes were made since my last summary. - [x] The changes remain concise; I didn’t add any extra implementation beyond the classifier/test updates and the PR-description clarification, so there’s nothing extraneous I’d revert. New since my last summary: - I reviewed the Slack follow-up and my view is: - the current PR is a good minimal fix - Engel’s suggestion to prefer hard condensation for malformed-history errors is reasonable as a follow-up - but Calvin’s concern is important: doing that without strong logging / observability / loop protection could hide the underlying race or event-stream corruption bug So my bottom line is: - this PR should stay as-is as the immediate fix - a follow-up PR could make malformed-history errors go straight to hard reset, but only if we add safeguards so we don’t mask root causes or create retry loops.
Final summary: - [x] Clarified the implementation and PR text to make explicit that this change does not guarantee an immediate ā€œhardā€ condensation. - [x] Updated the PR description to say the intended behavior is to capture these malformed-history errors and retry with condensed history, with regular condensation first when available and hard reset as a fallback. - [x] Posted the requested PR comment verbatim, with the intro ā€œHi, I'm OpenHands-GPT-5.4.ā€ - [x] Included my thoughts on the Slack follow-up discussion exactly as requested. Conciseness: - The new work was limited to communication updates only: clarifying wording and posting the requested PR comment. - No additional code changes were introduced beyond that.
Final summary: Since the last summary, I: - posted the requested PR comment verbatim, with the exact wording you asked for and the intro ā€œHi, I'm OpenHands-GPT-5.4ā€ - answered your logging/observability question based on the code paths already examined Answer to your question: - No, we do not currently have enough logging to fully address Calvin’s concern. - We have some low-level debugging support: - failed LLM calls can be captured by telemetry error logs with exception type/message/traceback and request context - But we do not have enough high-level structured observability to avoid hiding this class of bug when recovery succeeds: - the malformed-history error gets remapped into a generic
LLMContextWindowExceedError
- the agent logs a generic ā€œcontext window exceeded, triggering condensationā€ warning - if recovery succeeds, there may be no top-level conversation error visible to users/operators - So: - manual forensic debugging may be possible sometimes - but production visibility/tracking is not sufficient - Calvin’s concern is valid: successful recovery can make a real underlying bug look like a normal condensation event Checklist: - [x] Posted the requested PR comment exactly as instructed - [x] Answered whether current logging is sufficient - [x] Focused only on new work since the last summary - [x] No extra code changes were made in this final step Conciseness: - The new work was concise: - one PR comment - one direct analysis of the current logging behavior - No extraneous repository changes were introduced in this step.
w
Ideally this also isn't a context window exceeded error, we should make another exception class and have another case in the agent logic catching them.
šŸ‘ 1
c
Done
n
Done. What changed: - added a dedicated
LLMMalformedConversationHistoryError
- split malformed-history detection out from
is_context_window_exceeded()
- mapped Anthropic broken
tool_use
/
tool_result
errors to that dedicated exception - added a separate
Agent.step()
catch path with distinct logging - recovery still uses
CondensationRequest
- logs now explicitly say this is malformed history / likely upstream event-stream or resume bug - updated tests accordingly - updated the PR description to match the new behavior and rationale PR: - https://github.com/OpenHands/software-agent-sdk/pull/2613 Current PR state: - latest pushed commit:
bd3569e5
- requested review from
all-hands-bot
- bot review landed on the updated commit - I babysat CI until it went green - merge state is still
BLOCKED
, which looks like normal repo merge requirements rather than failing checks And yes — for the rest of this session, if you ask me to summarize, I’ll just repeat the last thing I told you.
w
I think it's gone sorry!
c
Do we still need the PR? It would be an extra catch, I guess. Or did something else fix this kind of edge case?