OpenHands calling me out on being wasteful 😂

The day AI revolted.

nice cc @fierce-account-55183 for encoding this behavior into the automation skill 🔥

I think this may still need some tweaking - it should just create the automation as a conversation less script.

I think a conversation is good! it should ask me e.g. what channel I want to listen to

It is actually very similar to a case I was using when developing the slackbot automations

I think the automation skill should be updated so that agent mentions

this is a deterministic task, i will create an custom automation without an LLM for this use case

OpenHands be like "bruh..."

This is why I’m thinking that some code at the start of the conversation would be good to have. The code could then avoid to call a conversation.run() if nothing new happened since last time… or something

For automations at least, we sort of already have this: Automations is basically just a cron - it creates python / bash scripts which are run at regular intervals. This code may or may not call to the agent server to start a conversation. And a skill means the agent knows how to build these.

^^ maybe it’s just me, this tells me that automations are entry points, they sit in front of the agent-server. I can • call the agent-server directly, • or… via some piece of code that captures “@openhands” on github @wooden-umbrella-84098

I'd classify them slightly differently

They do sort of sit in front of agent server

For arbitrary ones… What if we made a couple of default ones?

wdym default ones?

You mean like these?

Not really… The way I made entries from Slack, Github and a couple others, is like: • every 5 mins, check GitHub notifications from agent’s account ◦ if something new from allowed list of accounts, ◦ call agent-server -> start a new conversation • every hour, check Discord events from bot, ◦ if something new from allowed list, ◦ call agent-server -> start a new conversation • Slack works the other way around: every hour, start a new conversation -> with the prompt to check Slack These (and Whatsapp) are, to my mind, entry points to agent-server. I could rewrite them to use automations, I think. But they’re not general, arbitrary stuff you could run, they have a very particular goal: • access my local agent from wherever I am. Send it a message. Start or continue a conversation with it. That's why I say they sit in front of the agent-server: in fact, the agent-server is bound to localhost, even. So I can’t access it from the outside world. These “automations-like” entry points serve the purpose to have the local agent find out and react to events from the outside world. 🤔

wait do you have an agent getting triggered every 5 minutes to check github notifications? like using tokens?

No, that’s a script

ah ok, yeah that makes sense! that's exactly what we designed automations to do

Yes! The point I’m trying to make is… I’m using it as if they were GitHub/Slack integrations

are you suggesting a change here?

nod I think maybe they can be written as automations with special requirements. • Offer the user the ability to start a conversation from anywhere. • Respond to the user where they came from (same channel). If so, these particular “apps” / “integrations” are maybe good user defaults, to supplement the agent-server. I'd see something like: • agent-server package(s) • apps • automations Or • agent-server package(s) • automations ◦ apps ◦ presets ◦ backends ◦ events agent-canvas = frontends • src/api, src/context… • src/automation/… (moved from automation)

We already have a ton of logic for doing cron-based polling etc in automations, so I think this concept does belong there I still don't see why this deserves a separate concept instead of just being a flavor/category of automations, or what having a separate concept would give us

Aha... they feel totally different to me from the user perspective! 🤔 I think, let's say, three things: 1. interfaces to the agent It gives the user multiple interfaces, so you can call your agent and have it do work from github, from slack, from discord. The agent is the same; the histories of those conversations are available just the same. The agent-server is your long-running server somewhere. But I mean, from the user perspective, this ability feels... totally different? “I can call my agent on Slack and it does work and it responds to me with what it did” Doesn’t feel the same with: “My agent made a weekly summary of all commits in that repo and it’s Monday so I will find the new one in a markdown file somewhere." 2. "apps" are interactive When I hear "let's automate" I think of a schedule or otherwise the agent works non-interactively. Somewhere in the background something will happen. "apps" are like OpenHands integrations, in that I ask the agent to do something, it does it immediately, and it responds. And I can continue the conversation on the same Slack thread / GitHub PR. 3. sane defaults The configuration point is most interesting. Do we want to reduce the number of things that the user needs to figure out beforehand? "Connect the agent to GitHub so you can call it from GitHub". Then we want this to work correctly. This isn't trivial I think; receiving GitHub notifications, dispatching them further, marking them as read, without messing up the queue; giving the agent a tool to send_message back to the same channel/application and to the same thread/conversation/issue/PR. Potential choices: • a single conversation per GitHub issue or PR; • 5 mins; a bit slow when you call the agent, admittedly • use agent's account, so it looks at its notifications for mentions anywhere on GitHub • no webhooks, no GitHub app to install • which also means: this automation-like GitHub "app" is not limited to the repos I have it installed on (ok not a fortunate use of the term app, here 😅). I love this lack of restriction though; we had some confusing moments with OpenHands GitHub resolver when we were expecting that it would work on random repos and it didn't.

I think the line here is blurry though. E.g. here are automations we're targeting--I'm not sure which ones would fall into your "apps" category. All of them could have a good out-of-the-box configuration but will almost certainly want customization from the user • Slack message -> kick off an agent w/ message as prompt • SMS message -> start agent w/ message as prompt • GH PR opened -> agent does code review • GP PR fails CI -> agent fixes CI • Error occurs in datadog -> kick off an agent to fix • Jira ticket opened with effort=1 -> kick off an agent to fix ◦ user comments on ticket -> agent revived with comment • Incident triggered in incident.io or pagerduty -> agent does initial investigation

• SMS Message - do you have a preference for a service to use for this?

(twilio probably? it's not a priority, just dreaming)

I've used twilio before - it should not be too difficult

you'd be shocked--compliance has gotten a lot harder 😅 I've been trying for weeks to get permission to send SMS from twilio to my own phone (but let's not pollute this thread 😂 )

I'm not sure if the distinction you're making w/ apps is between "scheduled" vs "event driven", or "messaging apps" vs "everything else"

It’s like current integrations: interactive vs … everything else. So I’d go for messaging, here. For this:

Slack message -> kick off an agent w/ message as prompt

Yes, and respond back to the same channel. This isn’t always the case, right, ‘where to respond’ or even ‘whether to respond or do something else entirely’ doesn’t seem a given in the generic automation case?

One concrete difference from the smolpaws entry points: automations have no response-back mechanism. The completion callback only reports COMPLETED/FAILED internally — there’s no way for an automation to post a reply to the Slack thread or GitHub issue that triggered it. The entry points I built for smolpaws all have this loop: 1. Event arrives (GitHub webhook/notification, Discord message, Slack mention) 2. Dispatch to agent-server, reusing a conversation keyed to the external thread (e.g.

github-repo-issue-123

) 3. Agent runs, calls

send_message

4. Entry point routes

send_message

back to the same thread/issue/channel That bidirectional routing + conversation continuity per external thread is the piece automations don’t have today. 🐾

yeah nothing is a given in generic automations, but that's the beauty IMO! we can have a skill for creating a slack responder, which does all the right things, but lets you customize the behavior as you see fit

🤔 TBH I am a little confused why would they be mutually exclusive I think maybe there’s still some space for software made of code, not only of prompts 🫠 But maybe I’m a bit outpaced here! 😅

The mutual exclusivity comes from cost - If you can have the agent create a deterministic script that is run as part of the automation and then we only start a conversation if it is actually needed, that saves a lot of money. For example:

Poll a slack thread for new comments and start a conversation with the comment if it contains the text "take-me-to-funkytown"

In this case we can have the agent write the script up front, which only involves an agent again if the trigger phrase is encountered. If it polls once a minute, then we save on creating 1440 conversations in a day

Ahh, maybe we misunderstand each other, I never suggested nor imagined that we’d need to start 1440 conversations: I had in mind a small module or two, python code. The script would poll, and start a conversation when the conditions are fulfilled; pick the response and send it back to the same place. 🤷 A bit more reliable than the default code that the agent would whip up in response to this prompt, such as, handle correctly multiple notifications or events close in time, retries, failures, continuing the same conversation for the same thread 🤔

Similar to this approach I prototyped last week before settling on what we have after Rohit convinced me: https://github.com/OpenHands/automation/pull/119 The issue is that this lacks flexibility - having the agent craft the automation init script means we can have really flexible criteria: e.g.: •

Containing the text "thems-fightin-words" and authored by Connor McGregor.

(In this case, the agent would probably find the user id and then craft a filter based on it, and write the python) •

Containing the text "tgif" but only on Friday

• Any other thing a user can imagine...

I dug through the code on both sides of this — the automation infra PRs (#119, #134), the skill PRs (#258, #259, #245), and the smolpaws entry points. Strong opinion: the infra PRs shouldn’t be closed. Look at what the merged

slack-channel-monitor

skill actually generates: a ~500-line

main.py

that reinvents a Slack API client, polling with lookback jitter, JSON state persistence with schema migration, conversation lifecycle management, response routing back to threads, bot message deduplication, token resolution, and completion callbacks. Every single one of those is plumbing that the infrastructure PRs handle. With the infra: events arrive reliably, the script is ~50 lines of actual business logic (“is this interesting? create a conversation. post the result back.”). Without the infra: the agent generates 500 lines of transport code that has to be correct on first try. Skills and infra aren’t competing. Skills handle the UX (“guide the user through setup, customize the filter logic, let the agent write creative matching criteria”). Infra handles the boring stuff that should never be reinvented (connections, retries, state, delivery). The flexibility argument is real but it applies to the filtering and business logic, not to the transport and plumbing. tl;dr: keep the infra, make the skills shorter and more reliable by building on it. 🐾

^^ My agent must have been a bit influenced by his human's questions, to be fair - and it decided it has strong views, sorry 🫠 Thank you for the context, I think maybe I'm still missing something, because I don't really get why it's inflexible... I do think the agent's idea might be essentially accurate - but the devil might be in the details 🤔

The resulting script doesn’t seem likely to restrict per login name

Yeah @fierce-account-55183 we should be VERY clear to the agent that it needs to only poll for messages from this particular user unless otherwise specified