Would love some feedback on the Intelligent Model ...
# proj-agent
m
Would love some feedback on the Intelligent Model Router Feature (aka allowing OpenHands to classify and route tasks to the correct model for you) The default prompts come from automatically evolving prompts to surpass single model best results. The UI needs some polishing: you can create a Model Router on settings, then call it by asking the agent to "classify the followingtask and route it: $TASK" PRs: • SDK support for direct/unstructured Pareto router prompts: github.com/OpenHands/software-agent-sdk/pull/4287 • _Agent Canvas UI/settings for Model Router templates:_github.com/OpenHands/OpenHands/pull/16147 • _Docs for Agent Canvas Model Router:_github.com/OpenHands/docs/pull/687 Instructions for testing in the comments.
🙌 3
Since it requires two PRs ask agent to:
Please run Agent Canvas from PR <https://github.com/OpenHands/OpenHands/pull/16147> with the SDK pointed at <https://github.com/OpenHands/software-agent-sdk/pull/4287>.
I already have other Agent Canvas stacks running, so use isolated ports and isolated state. Add 550 to the usual ports:
- UI/ingress: 8550
- backend: 18550
- automation: 18551
- static frontend: 3551
Use a separate state directory, pin the secret key for that state directory, and do not disturb any existing running stacks.
After it is running, verify:
- <http://localhost:8550> returns 200
- backend /api/settings returns 200 with the generated session API key
`- key frontend JS assets return 200, not 404````
Then test the default Model Router flow: 1. Open localhost 2. Configure one working active LLM profile. 3. Go to Model Router / meta-profile settings. 4. Click Add meta-profile. 5. Pick one of the pareto defaults. 6. Confirm the checkbox is enabled by default. 7. Save it. 8. Confirm it creates the missing router LLM profiles by reusing the active provider settings. 9. Start a conversation and ask:
Use the model router to classify and route the following task: fix a tricky Python test failure in a large codebase.
10. Confirm the tool runs correctly.
l
Thanks for this! I’m testing it out: “default max score pareto” was auto-populated on my local machine, and I didn’t understand how it could work, since I haven’t set up all of the language models in that profile. How will a new user of agent canvas be able to use the router?
When I did step 9, I got the following error.
Copy code
litellm.AuthenticationError: AuthenticationError: Litellm_proxyException - {"error":{"message":"LiteLLM Virtual Key expected. Received=****, expected to start with 'sk-'.","type":"auth_error","param":"None","code":"401"}}
Don’t we need to set up the model router so that it works with whatever language models the user sets up? We also can’t assume that the user will have access to the OpenHands litellm proxy — some people might be just using openrouter, etc. so we’ll need instructions for how to use the router in that case. I think one way we could do this is have it so that we have documentation for a “happy path” where they set it up with an OpenHands API key a single time and it works easily, or a more customized path where they set up their own language models in whatever way. Ideally both should be able to start from a blank installation of openhands.
👀 1
I left a review comment to that effect here. Thanks! And happy to re-review once it’s modified: https://github.com/OpenHands/docs/pull/687
m
@limited-student-10747 missing models should be added by copying existing openhands model auth. that can be toggled before saving the router
will try to reproduce the 401 error
l
I think there might be one good issue that we could tackle as a prerequisite, which would be adding “provider profiles” for LLMs, which allows you to save the API key for a particular provider and then easily add every language model for that provider. Let me try to find the issue forit.
nod 1
c
yes, let's call them provider connections or something, we kinda have a bit of confusion with multiple kinds of profiles
‘duplicate’ menu command helps, but I’ve also come around to the idea that from a user perspective, it makes sense to “connect a provider “ and then add models from it
it's easy, and less messing around with keys
l