Should we combine LLM profiles and Agent Profiles? I could sort of imagine reusing the same LLM Profile across agents, but it'd be a much simpler mental model to just set one thing up, and have one picker in the UI

I was thinking per conversation Agent Profile picker + LLM Profile picker that can override whatever the agent would default to. Then long term I think Agent Profile has LLM Router which uses LLM Profiles available to route to. If no user override in a conversation the Agent Profile routes automatically to the LLM.

Please see also the message above this

My instinct is no, for the same reason we shouldn’t combine MCPs with agent profiles. agent profiles have a lot of “variables by reference” to things like MCPs and LLMs, so you can change the underlying LLM profile or MCP, and have that change be uniform across all the agents that use that LLM or MCP

Having both agent and llm profiles look alike, is confusing me a lot TBH • for some reason, at convo start, it’s agent; at convo runtime, it’s llm 😅 • when I open Settings, I don’t see any clue that making a change would affect an Agent profile (which is okay imho if: Agent Profiles are disabled, and I’m running off a default one) • Agent Profiles … imho need their own UI somehow, or UI clues

@breezy-kilobyte-39430 🥹 ^, any ideas?

> for some reason, at convo start, it’s agent; at convo runtime, it’s llm yeah, the reason was you can't switch agent profile at runtime, but you can switch llm profile / model_name (acp) at runtime Now it's not because we can that we should. 😅 > But from a visual user perspective, I want to switch LLMs, not define Agent Profiles… why do they help me? You want a specific agent for helping debug prod issues with read-only tools and with access to "powerful" secrets. (datadog or even kubectl access) • you want it to have read-only tools and no editor • you dont want other agents to have access to those secrets

Aha 😅 yes, we need to give it some interface that doesn’t get confused with llm... sorry, Devin, I think if we want both in the same time it's possible but idk if a bit complicated... Maybe we can test it without jumping in meaning of the same UI element 🤔 I would suggest maybe we could: • define agent profiles and their REST API - maybe in the design doc we were mulling over ◦ this is what my agent hit, with the tool-less request -> agent-friendly is just good ole API design • undo the "jump in meaning" for now -> keep the dropdown for llm profiles, think about adjusting them rather than making the cognitive toll on the user, idk, more complicated • put the agent profiles under a feature flag until we figure out how the visual experience is ◦ maybe a flag accessible to the user, e.g. a checkbox somewhere in Application settings? ◦ just so everyone can play with it and we may get more feedback, idk

I think agents and llms are inherently different, just like picking a mode is different from picking a model. @hundreds-twilight-32833 I don't think we want to merge them into a single dropdown.

How about the general ui/ux for agent profile + LLM selection? We can look at some different strategies from other tools that have vendors+lots of LLMs available. Not sure if you have any general flow thoughts or strong opinions @breezy-kilobyte-39430

+ menu definitely seems better to me!