https://docs.all-hands.dev/ logo
Join Slack
Powered by
Hey folks — I’m wondering if there’s a recommended...
# questions
f
Hey folks — I’m wondering if there’s a recommended way to integrate a CLI app as a custom LLM tool. For example: say I have a CLI tool that’s not very popular / well-known, so agents don’t automatically know to call it via the generic terminal tool. I’d like to wrap it as a custom tool in openhands-sdk. The benefit would be that when the agent uses this custom tool, I can see it explicitly in the tool-call history — not mixed in with other generic terminal commands. Right now my approach is just spawning a subprocess in Python every time I need to call the tool. But that feels somewhat inelegant. So I’m asking: is there a recommended or more “proper” way to wrap a CLI-only app so it works cleanly as a first-class tool in openhands.
✅ 1
nvm, just found u can use the TerminalExecutor for actions in your custom tools.😇
🙌 1
l
Great, glad this will work!
f
Hey guys, I encountered this problem once again. My concern is different this time. Of course I can make this CLI app a tool and execute it with the 
TerminalExecutor.
However, for cases like
Copy code
rg -l "TODO" . > ripgrep_result.txt
or a pipeline like
Copy code
my_custom_tool xxx | grep "" > result.txt # keeping this in a local file since it is relatively large.
Merely wrapping a CLI tool and execute its args in the 
TerminalExecutor
is not sufficient.
So what I’m thinking is that maybe we could add a special decorator for these tools that marks them as “virtual tools”. These tools don't have be implemented like other custom tools, the user only needs to provide enough information for the model to know this tool exist in the terminal. The model would still see them as tools, but it would also know that they’re implemented via the terminal tool and the only way of using these tools is to use the 
Terminal
tool and pass in corresponding args. Then, these virtual tool calls could be surfaced explicitly, instead of being mixed in with other generic terminal commands. Do you think a feature like this would make sense? If this sounds like a feature worth exploring, I’d be more than happy to help implement it or contribute a PR.
l
Thanks for the idea!
Then, these virtual tool calls could be surfaced explicitly
To clarify, how would you actually do this? And would this only be a difference of how it looks in the interface? Or would there be some other difference from how it’s used in the actual agent?
f
Thanks for the follow-up question — that helps me clarify what I had in mind. From an implementation point of view, I was thinking of something like this: • These “virtual tools” would still be exposed to the agent as normal tools (with their own schema or the agent can learn their usage by calling custom_tool --help), but their description would explicitly say something like: “This tool can only be used via the terminal as a CLI command.” • The agent would then be encouraged to call them through the terminal tool rather than via a direct function call. • As a fallback, if the model still tries to call the tool directly as a function, the tool implementation would just return something like 
stderr: "This tool can only be called in a terminal"
so the agent can correct itself. So in that sense, the agent logic stays essentially the same — we’re just giving it one more CLI capability that it’s aware of, plus a clear error signal if it uses it incorrectly. On the UI side, the idea would be: • when the terminal tool runs a command that includes one of these “virtual tools” in the command line, the UI could detect that and surface it as a call to the corresponding virtual tool, instead of showing it as a generic terminal command. This is roughly what I was thinking. I’m not sure if this fits well with the current design philosophy of openhands, so I’d really appreciate your thoughts or suggestions on whether this makes sense, or if there’s a better way to achieve something similar.
Another (maybe simpler) way to think about this is as an extension of the existing terminal tool. For example, the 
TerminalTool
could expose a helper like:
Copy code
TerminalTool.extend_cli_tool(
    tool_handle="custom_tool",
    description="..."
)
Internally, this would just update the description of the terminal tool by appending a section like: ### Other CLI tools #### custom_tool ... That way, the agent still only sees a single terminal tool, but it is explicitly informed about additional CLI capabilities and how to use them. This feels like a relatively lightweight and efficient approach to me
l
I think there’s two problems you want to solve here: 1. Teach the agent how to use the tool through bash — for this you can use skills: https://docs.openhands.dev/overview/skills 2. Surface it in the UI -- this is not implemented, but “terminal tool with custom observation parsing” could be an interesting prototype. For this, I would suggest sub-classing the terminal tool and overriding the observation parsing part to return different observations depending on what bash command was called. If you have an example of how you can do this for your use case, we could then maybe think of how to generalize it.
f
Thanks so much for the detailed suggestion! I think you're absolutely right that there are two separate problems here. Teaching the agent how to use the CLI via bash is one part, and skills look like a very clean way to do that — I’ll give them a try and see how far they get me. For the second part, surfacing things in the UI, I’ll start by experimenting with a terminal tool subclass plus custom observation parsing. I’ll see if I can put together a small example for my use case and share it back in a PR draft maybe.
🙌 1
Open in Slack
PreviousNext
Powered by