https://docs.all-hands.dev/ logo
Join Slack
Powered by
Are we actively developing plugins still? I'd love...
# proj-agent-canvas
w
Are we actively developing plugins still? I'd love to get a preview if so
m
Hello @wooden-umbrella-84098, Are you referring to the Plugins page? We have already implemented this feature, and the code is currently available in the main branch. However, the next version of agent-canvas that includes this update has not been released yet. Please feel free to refer to the video below for more details. Let me know if you have any questions or need any additional information. Thank you very much! thankyou
plugins.mov
w
Ahh I think I confused "Plugins" here with the idea of adding custom UI elements @hundreds-twilight-32833 I know you were interested in this one, guessing we haven't started it?
thankyou 1
h
I've been chipping away here and there on the weekends. I just added my most recent PRD to the shaping issue github.com/OpenHands/agent-canvas/issues/481 that @calm-artist-7206 has had some input on. I think @curved-angle-80449 also had a poc he did over a weekend.
I think we could make it happen pretty quick but it needs a technical lead to make a blessed path that aligns everywhere
w
Let me tell you my use case and you can tell me (a) if this supports it and (b) if it's the right thing to target I created a small app for generating blog posts, which uses the agent-server as a backend. I give it a topic, and 1-2 paragraphs of narrative. I can then kick off an agent to do research and put together a blog post. I can make comments and kick off another agent to address them. I'd love for this app to just appear in my left-hand nav Ideally I could use a slash command to do this in one shot: 
/create-app that allows me to use agents to create blog posts from a set of keywords and a description
. OpenHands pulls in a skill that builds the app as a standalone React app, links it in, connects it to the right agent-server backend, adds a nav item, etc
🤔 1
It's tricky because it has its own backend (it needs somewhere to store ongoing work)
But "build agent-powered apps" is a really powerful story
h
100000%. that is where my head is at with this too. We could put storage by default in .openhands but I don't know if that is the right call. It could just be a workspace folder as well. I had the side nav story working when working from main but I don't know if it would work with a global npm install. I don't know if that branch would still work since its been a few weeks.
w
I would want my plugin to go in version control, so I'd want it to be its own git repo. Storage I'd take care of (I'm using sqllite)
thinky spin 1
Also looks like you're building the apps together, which is tough--I'd want someone to be able to 
npm install
and then tell openhands 
/create-app
Ideally that'd just populate a JSON file which adds buttons to the left nav
h
It might be better to go in reverse. Build the extension with the shape we want, then make it work. And then add a create-app skill that'll do it
nod 1
w
At minimum, it could just be something like:
Copy code
- title: Blog Generator
  icon: pencil
  route: /bloggen
  tag: bloggen
  path: ~/git/bloggen/dist
- title: Vuln Resolver
  icon: bug
  route: /vulns
  tag: vulns
  host: localhost:3000
UI would just let you filter convos based on tag, and show a list of registered apps. User could be responsible for serving (e.g. I am using nginx to serve mine at bloggen.apps.canvas.rbren.io) or we could use a proxy to serve it on 
route
h
Here is an example of a nav page: github.com/DevinVinson/…/hello-page
Although the page content isn't in that example for some reason. Maybe lost switching/pushing branches
I think the work Hiep did on plugins would help here too. That built in some install/versioning bits
w
very cool
c
I have a branch of Agent Canvas that lets you add extensions to five so so different places--new panes, various context menu places, settings, etc. it can load plugins dynamically from a github repo or from an npm package and has same security semantics and guarantees as vscode...
h
Add a link on that ticket! It would be cool to see another path
c
Unlike y’all, I'm afraid I’m a bit confused here… I thought we want a thin UI, canvas = thin UI; and then, addons in canvas = UI-only e.g. addons are able to add UI elements, of course add to the left column
That's conceptually very different than “and have an sqlite db” 🙏 🫠
h
Once you open up a route any SPA could get dropped in realistically
c
Right… Maybe we should separate that, adding routes, from the other? • UI enhancements / customization • apps Or maybe that’s too old fashioned 😂
Also looks like you're building the apps together, which is tough--I'd want someone to be able to 
npm install
and then tell openhands 
/create-app
Just for clarity, what do you mean here by “you’re building the apps together”?
Also for clarity:
I created a small app that uses agent-server as backend
What does ‘app’ mean here, is it able to run without agent-canvas?
w
you’re building the apps together
the way I read the GitHub issue, you need to 
npm run build
on agent-canvas in order to pull in your extensions
What does ‘app’ mean here, is it able to run without agent-canvas?
Without the canvas frontend, yes, but it needs an agent-server
c
OK! So it is a separate 
app
, and then maybe “app” is well named; and then you’d like UI addons that allow to add things in the UI -> add a link to it?
I wanted to say that they are still two separate things; but maybe I’m wrong. One would expect more coupling than just a link
> you’re building the apps together
the way I read the GitHub issue, you need to 
npm run build
on agent-canvas in order to pull in your extensions
Actually, I think we are talking about two different things. “extensions”, or, as Devin named them, “addons”, are addons for agent-canvas. They enhance the UI, replace elements, add elements; they cannot run without agent-canvas.
The idea of ‘apps’ is different: I can run it, and not run agent-canvas. It’s just that I can also, if I want to, use the UI addons to link the two apps, that is, to make the app show up in canvas in some ways.
Rethinking the last paragraph (sorry, just trying to understand better): canvas UI needs some extensions points; a “simple UI addon” can plug into those and make stuff show up; they’re extension points (e.g. a manifest will register this or that code to be run on some event) If 
blogger
is a different app, how would it make stuff show up in canvas?
h
Maybe its worth some video demos to talk show and tell. The basic implementation isn't too hard to build but these kinds of details are where I've hit some hurdles
nod 1
c
Yes. Just to note, on another side of this: I actually do have 4-5 “_apps_” doing exactly like Robert says, they wrap the agent-server. And they’re completely separate from each other, as in, you can run one or two of them or all of them.
The powerful thing there is, the agent knows itself. It knows them. I can tell it to start one or the other or fiddle with them
And I’m thinking, the answer to my question might be super simple: we could say, OK, for the top left column in agent-canvas, we have a default list, and, when canvas starts, it could maybe look also in ‘apps/‘ directory; if the user has some apps there, read (predefined little format) their title, link, backend, and add them to the list.
h
I was having a separate conversation about something similar today. Essentially asking should a backend advertise its capabilities in the server_info. For this conversation, that would mean inside of server_info the frontend UI, agent canvas, would see "apps" and know how to handle them. Wherever canvas is installed as as backend it can tell another instance what UI will be available. That may or may not be fully relavent since I'm getting out of my comfort zone with server<>local backend setups but its what your comment made me think of 😄
c
Ah! I think you’re right. Backend comes first
6 Views
Open in Slack
PreviousNext
Powered by