https://livekit.io logo
Join Slack
Powered by
Can i override an LLM on the same agent between 2 ...
# ask-ai
c
Can i override an LLM on the same agent between 2 turns? I don't want to use agent handoff
✅ 1
t
Thanks for reaching out, Yousri. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You're asking whether you can override an LLM on the same agent between two turns without doing an agent handoff — yes, you can. You can change the agent's LLM dynamically via the session (for example by setting 
session.llm = new_llm_instance
or providing a different 
llm
when constructing the agent/session). This lets the agent switch which LLM it uses mid-session without returning a new agent from a tool call or performing a handoff. Would you like a short code example for the language/runtime you're using (Python, Node, or other)? Sources: Agents and handoffs | LiveKit Documentation, with interactive elements
c
Are you sure? property 'llm' of 'AgentSession' object has no setter
t
Good catch — you’re right that 
AgentSession.llm
has no setter. You cannot reassign 
session.llm
directly. You can still change which LLM the agent uses between turns without a handoff by overriding the agent’s 
llm_node
and choosing the LLM dynamically there. Example pattern (Python-like):
Copy code
async def llm_node(self, chat_ctx, tools, model_settings):
    if should_use_large_model(chat_ctx):
        llm_instance = inference.LLM(model="openai/gpt-4.1")
    else:
        llm_instance = inference.LLM(model="openai/gpt-4.1-mini")

    async for chunk in llm_instance.chat(chat_ctx=chat_ctx, tools=tools):
        yield chunk
If you prefer replacing the whole agent instead, use 
session.update_agent()
to swap to an agent preconfigured with a different LLM. Which approach would you like a full example for (override 
llm_node
or 
session.update_agent
)? And which runtime are you using (Python or Node)? Sources: Agents and handoffs | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation, with interactive elements
2 Views
Open in Slack
PreviousNext
Powered by