Hey again — I saw the issue “[Feature] Trajectory ...
# questions
f
Hey again — I saw the issue “[Feature] Trajectory Replay” (#6049) in the OpenHands tracker. That suggests replaying past trajectories is already on someone’s mind. Given that: - OpenHands uses an event-sourced state model and records trajectories / logs when running agents (agents implemented with the openhands sdk with log_completions set to True). - The architecture supports different runtimes and a GUI front-end / web UI. So I think it should be technically feasible to add a “import past trajectory / log + replay in GUI” mode. What do you think — is this still on the roadmap?
c
You're right, and import and replay in the Web GUI has been supported in the past, actually, but I think it was hidden under a feature flag, and at some point it may have broken unintently. Importantly at the moment, though. Please allow me to try to clarify a few things, because it's a bit chaotic right now, sorry 😅 OpenHands is transitioning from the V0 version (0.5x, 0.6x), to OpenHands V1. Version V1: • is built on agent-sdk • the CLI app is on V1 • the GUI is not on V1 by default, unless you use a feature flag I believe • in V1, the concept of "runtime" is gone, or at least local runtime is not the same. Agent-SDK runs the agent loop fully. The REST interface to SDK's agent-server works, and of course, it's open source, if you'd like to dig into it!
f
Thank you so much for the clarification! But I'm a bit confused about the openhands cli and the https://github.com/OpenHands/OpenHands/ repo. I think the OpenHands you are referring to is OpenHands/OpenHands right? Because I saw 3 weeks ago it bumps to v1 and released the first v1 version 1.0.6. But when I run pip install openhands it's installing the openhands-cli I think, so I'm a bit confused about what are the relationship between openhands-cli and OpenHands/OpenHands repo at the moment. From the repo, my guess is that the OpenHands/OpenHands repo is mostly the local GUI and its cloud deployment?
c
Yes, exactly
The 2 repos, the CLI app, and the web app-server, are completely separate, The CLi installed from pip lives in its own repo now:
They are both interfaces to Agent-SDK
and, on V1, they actually use agent-sdk as a dependency
f
Cool, thanks for the extra info. Just to clarify my use case — I’m mostly using OpenHands(openhands-sdk) as a research tool rather than for production(openhands-cli). I often build custom agents and run a lot of experiments, so I end up with tons of rollout logs. Reading them directly is pretty hard because the volume is huge and the raw JSON is not very readable. That’s why having a local GUI without Docker + a “log replay / import logs” feature would help a lot. I’d love to load my existing logs into the GUI and inspect the agent’s behavior step by step. But since the local GUI currently relies on Docker, that part becomes tricky on HPC/cluster machines where Docker isn’t available. If the local GUI could work smoothly without Docker, and ideally plug in more naturally with the raw SDK (e.g., directly load trajectories / logs from SDK runs), it would be perfect for research workflows. Happy to help or prototype something if that direction makes sense.
c
Thank you! Just to clarify, trajectory replay from the issue above meant to actually re-execute the past actions in a new (docker typically) runtime environment. If it is what you'd like to achieve, afaik it's not implemented right now in agent-sdk (we have rebuilt/ported a lot, but not quite everything that was implemented in V0), but I think it would be awesome
We have somewhere a script that allows visualizing conversations events, it loads json files just to see them and check what happened visually, I think this one: https://github.com/OpenHands/software-agent-sdk/blob/main/scripts/conversation_viewer.py
It's more of a debugging script, tbh
f
oh it seems like I have a misunderstood here. The "logs import" that I wanted was just a simple loading the logs from the logs_completion_dir into the beautiful GUI so I can view them easily. Thank you so much, I'm gonna give this conversation viewer a try.
p
https://dagshub.com/ might be a good data viz UI for aggregate metrics from all your experiments or you could use https://dvc.org/ directly for the various experiments you run, if you want more lower level control this way you would branch, build, run, measure; and hopefully can do it without app-level trajectory control
👀 1
f
Thank u so much, I'll look into this.
c
Indeed it seems to me like there are multiple ways to do this, whatever you wish really. You could also ask the agent to improve one of these scripts or applications as you see more fit
f
I’m trying to use the upstream
scripts/conversation_viewer.py
, but I’m not totally sure how to point it at my own data. The viewer seems to expect a folder like: conversation/ base_state.json events/ *.json My existing artifacts are just LLM completion logs under
output/*Agent/logs
— basically JSONL from
log_completions
. I don’t have anything like
base_state.json
or the event-style structure, so the viewer just says “missing base_state.json” and nothing loads.
But it's ok cuz before knowing about the conversation_viewer script, I just had my agent made one script similar to this, and it's working well just less beautiful, my agent also build it on streamlit btw😂.
c
The log directory it expects is in ~/.openhands/conversations by default - hah, yep
Well, actually, maybe we should fix this, it has a hardcoded path 🫠
Copy code
ENV_ROOT = os.getenv(
    "OPENHANDS_CONVERSATIONS_ROOT", "/home/xingyaow/.openhands/conversation"
)
f
it's fine cuz it supports pointing to a new dir in the UI, but my logs folder doesn't have a base_state.json or events/ folder, that's why it won't work for me.
Is this from a deprecated logging format or am I missing something that caused my logs missing these base_state.json
c
Just to clarify, so you are entirely correct about log_completions, a bit of confusion is because there are 2 directories with logs, which the SDK makes (if enabled) • conversations and their event log - it's in
~/.openhands/conversations
, they are events and conversation state, saved in json • the prompts and responses to the LLM - in
log_completions_folder
f
oh I see
thank u for the clarification
c
Most welcome, so you might want both! 😅
f
FYI, this is the script I made for viewing the completion logs, do u think this is worth a PR?
c
Please feel free to
f
hardcoded path
Copy code
ENV_ROOT = os.getenv(
 "OPENHANDS_CONVERSATIONS_ROOT", "/home/xingyaow/.openhands/conversation")
Is this already fixed? The version I'm seeing is
Copy code
ENV_ROOT = os.getenv("OPENHANDS_CONVERSATIONS_ROOT")
DEFAULT_CONVERSATIONS_ROOT = (
    Path(ENV_ROOT).expanduser()
    if ENV_ROOT
    else Path(__file__).resolve().parents[1] / ".conversations"
)
I was planning to fix this hardcoded path in my PR also, but it seems there’s already a decent default path now, so I guess there’s nothing to change.
c
Oh, sorry! It seems some older PR just fixed it yesterday when it was merged
😎 1
f
hahaha thank you.