https://docs.all-hands.dev/ logo
Join Slack
Powered by
Are we interested to answer this challenge? :sweat...
# proj-extensions
c
Are we interested to answer this challenge? 😅 cc @average-processor-75195 https://x.com/rhyssullivan/status/2069637281963708459?s=61 The verifier is doing great, albeit without recording: example using browser
a
hmm, like just point my openhands at it and have it try?
c
I could be wrong, the post has a lot of ideas, just, in my understanding, he is after the Verifier Agent, or maybe Reviewer+Verifier. The ability to verify itself and bring evidence. So yes, but I guess to make it a good proof we could make it also run an agent with pr-review skill? I mean this kind of verification maybe could be part of the PR? https://github.com/OpenHands/agent-canvas/pull/1440#pullrequestreview-4540026742
a
Here is a basic start, I will keep improving it: github.com/rajshah4/executor/pull/1 - includes mp4
Let me know what you think @calm-artist-7206 - I don't want to burn that many tokens, so this is a start. We could do a wider / deeper set of tests, but this shows the basic functionality with openhands doing all this. (I haven't really dug into this, this is basically the agent on autopilot at this point)
c
Admittedly I'm on the side of burning them tokens for stuff like this 😅 Isn't it amazing, Xingyao just posted about it, and it's super interesting to me that Rhys by accident named OpenDevin as the thing he's thinking about, when he wants an autonomous QA agent!
a
Yea, I didn't have to do much, openhands can cover most of the points that he wants
❤️ 1
c
I honestly believe the name recognition and standing out to the promise would be great Can we cut its verbosity? What LLM were you using?
If all we need is to perform the steps and let their code tell how to record it (is that what happened?), then concise answers from the agent would be nice
Lately I've been on a bit of spree to cut the output of the agent around the OpenHands repos too; it's just too much and human attention doesn't scale 😅
a
Yea, I can DM you the runbook for this
Newer PR for anyone else following along: github.com/rajshah4/executor/pull/2
2 Views
Open in Slack
PreviousNext
Powered by