:wave: OpenHands/OpenHands#16106 needs a human rev...
# proj-agent-canvas
f
šŸ‘‹ OpenHands/OpenHands#16106 needs a human review: fix: render the VSCode button on self-hosted (local) backends by @harish-chandramowli (the author isn't yet a trusted contributor, so OpenHands skipped the automatic AI review)
l
This one @calm-artist-7206 auto-reviewed and it says ā€œgood tasteā€ but it was not approved and merged. I’m not sure what the appropriate action is in this case? Maybe ask @calm-artist-7206 to approve? šŸ™‚
šŸ‘€ 1
(Generally, I think that if a bot leaves a positive review we should try to merge the PR quickly)
c
This is because I also have an automation, which reviews according to some criteria, and started to log them here: https://enyst.github.io/pr-reviews.html Yes… I think maybe we should get notified
l
here’s another example github.com/OpenHands/OpenHands/pull/16202
āœ… 1
c
https://openhands-ai.slack.com/archives/C06QT0AGY4W/p1785424200348479?thread_ts=1785421002.621759&channel=C06QT0AGY4W&message_ts=1785424200.348479
I think maybe we need a human in some/many cases, because… it depends what it does, or how much if affects, or how risky it is, or if and how it messes with code design Otherwise we risk getting into too much tech debt, up to the point, it seems to me, where the agent will not be able to get out. Maybe like we saw with ACP, idk, where the agent is not able to figure it out anymore. šŸ¤”
Please let me put this differently: a while ago, my impression became stronger and stronger that humans can’t and won’t review code anymore, but code design
l
I agree, but it’s a little bit frustrating for the code to get a positive review but then have it not be merged
c
As of now, my automation has instruction to use only ā€œcommentā€, not approve, not request changes. I can change that
Maybe, the way I’ve seen it done in other projects, the review should come from an automated account, and/or say in the first sentence that it is an automated AI review
l
I think it’s OK for it to only comment as long as a positive auto-review results into a human looking at it quickly after
Which also maybe we could ensure through a slack note
šŸ‘ 1
c
Do we have a slack token on the repo?
l
I’m thinking that we could do it through @modern-fireman-99755’s automation
c
I can make/run a workflow to post when the agent thinks it has a positive review
Or can we make a Slack app with a token here?
Right, so I need a token to post from mine, or Vasco’s should do double duty to pick up separately and post those PRs that have positive AI reviews
(or I can do it via
smolpaws
heartbeat, not sure I should, I keep it usually disabled because it’s very token hungry)
l
I think we can add that as part of Vasco’s automation, its seems like it can check at the same time
šŸ‘ 1
c
Yes, so maybe like: it gets told to split PRs in multiple categories, post separately those that: • have either a bot review • or the review says it’s automated (and it’s from a list of login names, I think) • and this last review is positive (or we can directly change and require it to be ā€˜approved’)
Category: ā€œHappy Agent, Needs Humanā€ šŸ˜