https://docs.all-hands.dev/ logo
Join Slack
Powered by
Why don't we have something like this?
# proj-agent-canvas
b
Why don't we have something like this?
I used to have this:
m
We do have this in agent-canvas
b
Yes, but you don't want to have to open the window, but see it directly
ah, btw, I mean only this:
and the important number is the total used context
nod 1
m
and when you pass the mouse over you have the summary with everything?
cc @breezy-kilobyte-39430
b
yeah, but we only need to see the used context or the %
it indicates if it's worth or not to continue the conversation
vs starting a new one
c
nod We could have some minimal element showing how much of context window was used by the last request
b
why do you need the last request?
I was thinking to have this, but per message
because sometimes you want to know that the agent messed up ingesting too much tokens
c
Implementation detail: because we receive the info from litellm
Which received it from the provider
if we compute it ourselves we’d need the right tokenizer
and could be off
m
you could also having it by message doing some math but it is more convoluted 🙂
nod 1
b
no no, we want the stats from the llm-server
i think...
c
The response from the provider typically says : input tokens, cache reads etc
b
Per message: input and output tokens is the interesting values (mostly the input tokens) Maybe something graphical:
but the general (not per-message) indicator is really urgent
nod 1
c
From what is the screenshot
b
because context-is-all-you-need 🙂
generated
And this is real
👍 1
@calm-artist-7206 actually, you are right, the used context per message is super useful to optimize the workflow
because sometimes the agent reads and writes too much, and you must simply tell him to not to do it
c
With Qwen 36B ?
b
27B
👍 1
way better
dense
MoE are not good for agentic
(compared to dense models)
*prompt and generated tokens
b
great feedback and examples
m
@breezy-kilobyte-39430 if you have a design for that, it should not difficult to implement ;)
b
or just tell your agent to copy the frontend implementation from hermes 🙂
this here:
(Hermes is MIT license)
b
WDYT? What's missing from this?
👍 1
b
Beatiful
b
@better-monitor-61654 are we missing anything else that should go in? I put Plan usage as a stub but we need to figure out where that goes in the meantime. @hundreds-twilight-32833
👀 1
b
I think it's fine. Maybe place the number inside the circle
9
and/or make the border of the circle a little more thick
2 Views
Open in Slack
PreviousNext
Powered by