An out-there thought experiment wrt using RL for skill optimization (because skills can so greatly affect agent behavior) If we can measure downstream outcomes from modifying a skill, is it worth training a small LM that can optimize prompts for the LLM? I suppose the same could be applied to user prompts (a small LM suggests prompt improvements, or clarifications from the user that would better steer the agent for the task at hand)?

@red-crayon-20970 and I are working on optimizing skills at the moment. So far it seems like one of the hard parts is actually measuring downstream outcomes -- each skill probably wants a different metric, and that often requires grabbing some kind of external information and linking it to the skill usage. What's the benefit of using a small LM to do that prompt optimization over something like Opus?

Ah gotcha The idea for using a separate small LM is that there are learnable parameters that try to maximize reward; to me using a big model to propose variants seems akin to throwing darts on a board (tldr RL vs evolution)

It's a good point though -- even though we're planning on using LLMs to optimize the skills, there's some big open questions: • What signal from historical trajectories can we detect/optimize that actually leads to better skills? • How do we actually use an LLM to do that optimization? Our current baseline is basically what Juan/Graham wrote about here: https://www.openhands.dev/blog/20260227-creating-effective-agent-skills Where we just get an LLM to summarize the relevant trajectories and propose a new skill text. But Anirudh is currently comparing that to a bespoke skill feedback loop tailored to the PR review skill, and so far it looks like tailoring your metrics/feedback mechanism to the skill itself is worth the effort.

nod things like reward inversely proportional to prompt length could be accounted for Thanks for taking the time to walk me through the explanations!

Glad to know other people are also thinking about skill improvement! We'll have something to show off soon, looking forward to more feedback

Nice idea! We actually had a paper where we trained a small LM to optimize prompts: https://arxiv.org/abs/2510.16932 It works well, but we tested it on a somewhat limited setting and I think many of the strong out-of-the-box LMs can also do it decently well to. And prompt optimization is not something that we do over and over again, so using a strong LM with something like GEPA would probably be fine too.