Hey, (Ref: <https://github.com/OpenHands/agent-ca...
# proj-agent-canvas
m
Hey, (Ref: https://github.com/OpenHands/agent-canvas/issues/570) I was playing around with agent-canvas and noticed that tool calls aren't displayed very well at the moment (see the video in the issue). I think presenting tool calls in a comprehensible way to the end-user not only enhances the user experience, but also makes it easier to debug, perform QA, and create good snapshot tests. For these reasons, I did a bit of scoping on how we could enable that. You can find all the details in the issue. Looking for feedback on the design, implementation, or just general thoughts!
c
Nice! I love the custom visualizers! You may laugh, but I actually come to this from the other side: I doubt people will want to look at the exact output of a tool call a lot. In fact, a nice UX might… not be the DX of a debugging tool. Hey, we have debugging or observability tools for that 🤔 I understand we may want to see details now, because we’re just building stuff; and it seems for ‘task’ and other tools that were new, we don’t have some nice shape. IMHO though, that’s different than both • what is nice for the user to see in this conversation - I… think it’s more of a conversation, than a debug session • what is the… how do I put this, what is the likely direction in the next ~six months for how people work with / expect from agents -> I feel strongly that it’s more for long running tasks, where you can’t keep up anyway… you need real observability, when you need it; or nice high-level view, when you don’t…
But that’s why I love the direction you took: give it to people. Give people the ability to make and add the visualiser they feel like
I could see an argument for going the other way, too: https://openhands-ai.slack.com/archives/C06QT0AGY4W/p1778946098902129
I mean, maybe we can split like: • a default visualizer is necessary ◦ we could keep it simple • give people the right tools, to create for themselves a visualizer ◦ cc @hundreds-twilight-32833 ‘s addons maybe ◦ without code modification • but even so, the events become too many ◦ easy to see in restoring long -ish conversations scenario • so whip up lazy loading of the full events payloads like @wooden-umbrella-84098 did ◦ then, the visualizers for the older events would show them like the latest events • or maybe go even further, and have a summary of “what went on before” ◦ presumably in conversations view ◦ or replace planner view with it (in some conditions?) ◦ in human, not in tool calling output, and not in full terminal output 🙏
👀 1
h
I've been thinking about a Timeline view for conversations. I wonder if we could reduce the agent tool calls and activity down to a single line of 'Thinking -> Working -> Worked for' and when you click that it opens a right hand panel with a more detailed and visual timeline of what the agent is doing. Maybe that would also improve the conversation load time and overall performance.
w
I would love to see thoughts explicitly displayed! I think we're just dropping them from the UI now? Here's what I have in my custom UI
m
we do have thinking already displayed as a tool call
👍 1
h
Current Thinking for reference