I am using LiveKit Agents (Python) and trying to dynamically update tools during a single turn using

update_tools()

inside a function tool. Here is my implementation:

async def set_or_switch_topic(self, topic_id: str, context: RunContext):
    current_tools = self.global_tools.copy()
    current_mcp_tools = self.mcp_tools.copy()

    for tool in self.tasks[topic_id]["tools"]:
        topic_tool = mcp.MCPServer._make_function_tool(
            self.session.mcp_servers[0],
            tool["name"],
            tool["description"],
            tool["inputSchema"],
            None
        )
        current_tools.append(topic_tool)
        current_mcp_tools.append(topic_tool)

    self.session.mcp_servers[0]._lk_tools = current_mcp_tools

    await self.session.current_agent.update_tools(current_tools)
    return self.tasks[topic_id]["instruction"]

Observed behavior: • • In the SAME turn: The LLM continues using the OLD tools • • In the NEXT turn: The LLM correctly uses the UPDATED tools My understanding from reading the source: •

tool_ctx = ToolContext(...)

is created once per turn •

tools = tool_ctx.all_tools

is read once inside

_llm_inference_task

•

llm_node = node(chat_ctx, tools, ...)

passes tools to the LLM once • There is no re-fetch of tools during the same turn My questions: Is it true that tools are snapshotted once per turn and cannot be updated mid-turn? Why does

update_tools()

not affect subsequent LLM calls within the same turn (even when tool returns output)? Is there any supported way to force the LLM to use updated tools within the same turn WITHOUT interrupting or starting a new turn? 1. Does the framework reuse the same

tools

variable across multiple

_pipeline_reply_task

calls in the same turn? 2. What is the intended mechanism for dynamic tool switching (AgentTask, interrupt, or something else)? Please answer with reference to actual LiveKit execution flow (ToolContext, _llm_inference_task, pipeline_reply_task).

Thanks for reaching out, Heer. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Thanks — that explanation makes sense. Now I want concrete implementation patterns to handle dynamic tool switching, given that tools cannot be updated mid-turn. My use case: I have multiple “topics” Each topic has its own set of MCP tools • I currently try to dynamically add tools inside a function tool using

update_tools()

• I want the LLM to immediately use topic-specific tools without waiting for the next user turn Please give different approaches with pros/cons and code-level guidance for each: 1. 1. Agent handoff approach How exactly should I return a new Agent or AgentTask from a tool? How do I pass updated tools and instructions? What happens to chat context? 2. 2. Superset tools approach How to register all tools upfront and restrict usage per topic? How to prevent the LLM from calling irrelevant tools? Any best practices (naming, grouping, tool_choice, instructions)? 3. 3. Interrupt + regenerate approach How to safely interrupt the current pipeline and trigger a new reply? How to ensure updated tools are used in the regenerated response? 4. 4. Hybrid approach (if any) Combination of the above for best UX (low latency + correctness) 5. 5. Recommended approach for production Given performance, latency, and reliability, which pattern is preferred? Important: • Please include minimal working code snippets for each approach • Tie explanations to LiveKit internals (ToolContext, pipeline, agent switching) • Highlight tradeoffs clearly (latency, complexity, reliability) Goal: I want the cleanest and most reliable way to achieve topic-based tool switching with immediate effect.

Thanks — glad the explanation helped. Below are concrete implementation patterns you can use to get immediate topic-specific tools available to the LLM, with code-level guidance, pros/cons, and how each ties back to LiveKit internals (ToolContext, _pipeline_reply_task, agent switching). I’ve kept each pattern minimal and focused on the mechanism you asked for. 1) Agent handoff approach (recommended when you need immediate tool availability) • Summary: Return a new Agent (or an AgentTask that switches agent) from your function tool. The pipeline will switch to the new agent scope, creating a fresh ToolContext for subsequent LLM calls so the LLM sees the topic-specific tools immediately. • Why it works with internals: The handoff ends the current agent scope and starts a new agent instance. A new `_pipeline_reply_task`/agent scope creates a new ToolContext reading the new agent’s tools, so the LLM will see the updated tools immediately rather than relying on the old, frozen ToolContext. • Pros: Immediate availability of new tools, clean separation of toolsets, reliable. • Cons: Slight change in conversation scope (new agent context), needs to manage chat context transfer, slightly more complexity to implement. • Chat context: You should pass relevant context/messages into the new agent so the conversation continuity remains. The new agent inherits or is seeded with the prior chat history as needed. Minimal pattern (Python pseudocode; adapt to your agent APIs):

from livekit_agents import Agent, function_tool, RunContext

class TopicAgent(Agent):
    def __init__(self, topic_id, tools, **kwargs):
        super().__init__(tools=tools, **kwargs)
        self.topic_id = topic_id

class RootAgent(Agent):
    @function_tool()
    async def switch_topic(self, topic_id: str, context: RunContext):
        # Build topic-specific tools for the new agent
        topic_tools = build_tools_for_topic(topic_id)  # return list of FunctionTool objects
        # Create the new agent with those tools and initial instructions if needed
        new_agent = TopicAgent(topic_id=topic_id, tools=topic_tools, name=f"topic-{topic_id}")
        # Return the new agent as the handoff result. The pipeline will switch agents and create a fresh ToolContext.
        return new_agent, f"Switched to topic {topic_id}."

Notes: • Ensure

build_tools_for_topic

returns proper MCP/FunctionTool objects (same shape as agent tools). • The new agent should be given relevant system instructions or you can pass the existing chat history into the new agent state so user context is preserved. 2) Superset tools approach (register all tools upfront; gate usage) • Summary: Register all topic tools when the agent is initialized so ToolContext includes the superset. At runtime, prevent the LLM from calling irrelevant tools by encoding gating logic in tool descriptions, tool metadata, or the agent system prompt and using function-level validation inside tools. • Why it works with internals: Because ToolContext is created at pipeline start, a superset ensures all possible tools are visible. You then rely on prompt/instruction and tool runtime checks to prevent misuse. • Pros: No need to change agents mid-turn; simpler state management and chat continuity; low risk of pipeline manipulation. • Cons: The LLM can still propose calls to irrelevant tools; you need strong guarding in tool descriptions, instructions, and runtime validation; increased token exposure (many tools in the LLM’s tool list may increase complexity). • Best practices: Give tools clear, disambiguating names and descriptions; prefix tool names with topic namespace; add

tool_choice

hints and a strong system instruction to select only topic-allowed tools. Minimal pattern:

# Build a superset of tools at init
all_tools = []
for topic_id in topics:
    for tool_def in topic_tool_defs[topic_id]:
        # name tools with a namespace to disambiguate
        name = f"topic_{topic_id}__{tool_def['name']}"
        all_tools.append(make_function_tool(name, tool_def['description'], tool_def['inputSchema']))

agent = Agent(tools=all_tools, system_prompt="""
You are to only call tools for the current active topic. Active topic will be stated explicitly like: ACTIVE_TOPIC: <topic_id>.
If a tool does not belong to the active topic, do not call it.
""")

# At runtime, set an "active topic" token in conversation so the LLM knows which subset to use:
# e.g., append a short assistant message or custom chat_metadata noting ACTIVE_TOPIC.

Tool-side validation (preventing misuse):

@function_tool()
async def topic_tool_wrapper(args, context: RunContext):
    active_topic = context.chat_metadata.get("active_topic")
    if not is_tool_for_topic(this_tool_name, active_topic):
        return {"error": "Tool not allowed in current topic."}
    # proceed with real tool behaviour

Notes: • Use clear naming like

topic_{id}__name

so the LLM and your validation logic can match tool-to-topic easily. • Include explicit instructions and an

ACTIVE_TOPIC

field in chat context so the LLM reliably knows which tools are allowed. • Add runtime validation to safely reject calls to non-active tools; that prevents side effects from incorrect calls. 3) Interrupt + regenerate approach (force new pipeline) • Summary: Abort the current pipeline safely (interrupt the running reply) and trigger a new reply so a fresh ToolContext will be created that uses update_tools() results. This is effectively simulating a new turn programmatically. • Why it works with internals: Canceling the current

_pipeline_reply_task

stops the pipeline which had the frozen ToolContext. When you begin a new reply cycle, the pipeline will build a fresh ToolContext reading the updated agent.tools. • Pros: Achieves immediate tool refresh while keeping the same agent; can be used when you must avoid agent switching. • Cons: Requires ability to cancel the running pipeline safely; not all environments expose a clean public API to do that; can be fragile; may result in partial responses being lost; potential race conditions. • Safety: Use public APIs to cancel if available. If cancellation requires touching private state it’s not recommended. Ensure idempotency so re-run doesn’t create inconsistent side-effects. Minimal pattern (conceptual):

# inside a function tool
await agent.update_tools(new_tools)

# Attempt to interrupt and request regeneration
# If agent API exposes something like agent.interrupt_current_reply()
# (Note: if no public API exists, this is not safe)
await agent.interrupt_current_reply()
# Trigger a new reply from the same user message or a synthesized follow-up
await agent.request_reply_with_context(synth_message="Please continue with updated tools.")

Notes: • Many frameworks don’t expose a safe `interrupt_current_reply()`; if LiveKit does expose public interrupt/cancel primitives, use them. If not, prefer agent handoff or superset approaches. • If you must use interrupt/regenerate, design side-effect-free tools or make tool invocations idempotent to avoid duplication. 4) Hybrid approach (best-of-both for UX) • Summary: Combine superset + agent handoff to get low-latency fallback and immediate strict switching. • Pattern: ◦ Register a superset of tools with clear topic names so the LLM rarely needs to switch agent. ◦ For deep topic mode where a dedicated set, tighter instructions, or different persona is required, perform an agent handoff. ◦ Use runtime gating and tool validation to ensure mis-calls are blocked. • Pros: Balance of immediate availability (superset) and correctness (handoff). Lower user friction; handoff used only when necessary. • Cons: Added complexity in orchestration; need to maintain both systems. Minimal Hybrid pattern:

# Init: superset tools, plus ability to create topic agents
agent = Agent(tools=all_tools)

@function_tool()
async def ensure_topic_mode(topic_id: str, context: RunContext):
    if light_switch_ok(topic_id):
        # just set active topic metadata for gating
        context.chat_metadata["active_topic"] = topic_id
        return f"Switched lightweight to topic {topic_id}."
    else:
        # need strong mode — handoff
        topic_tools = build_tools_for_topic(topic_id)
        new_agent = TopicAgent(topic_id=topic_id, tools=topic_tools)
        return new_agent, f"Entering deep topic mode {topic_id}."

5) Recommended approach for production • Short answer: Use agent handoff as the primary mechanism for immediate, reliable switching and the superset approach for light-weight, low-latency gating. Avoid programmatic cancellation unless a safe public API exists. • Rationale tied to internals: ◦ ToolContext is frozen per pipeline. Handoff triggers a new agent scope and new ToolContext, which is the supported way to get new tools into the LLM view immediately. ◦ Superset is simplest when tools are numerous but mostly read-only or easily validated, and it keeps the pipeline stable. • Tradeoffs: ◦ Latency: Handoff has small overhead to instantiate new agent context but is predictable. Superset has zero switching latency but may require more instruction engineering. ◦ Complexity: Handoff is conceptually cleaner; superset requires more guarding logic and stronger prompt engineering. ◦ Reliability: Handoff is more reliable because the LLM’s tool list matches the desired set exactly for that new context. Concrete implementation notes & tips • Tool naming: Namespace tools by topic with a stable prefix (e.g.,

topic_{id}__action

) so both prompts and your runtime gating can disambiguate easily. • Tool descriptions: Make, with interactive elements

You have whole repo access right? find some way anyhow such that we can update the tools mid turn. Think longer and give better response.

I hear you — you want a way to force tools to change mid-turn. I’ll be direct and thorough: with the public LiveKit Agents behavior, there is no supported, safe API to update the set of callable tools inside an already-running reply pipeline (the ToolContext is constructed once per reply). However, if you have full repo/control over your runtime and are willing to accept risk, there are several advanced, invasive options you could explore. I’ll list them, explain exactly where to change behavior (which internals to inspect), show conceptual code sketches, and call out the risks and alternatives. If you prefer a lower-risk path I’ll restate the recommended production approach at the end. Important: these invasive options require changing internal pipeline behavior, can break invariants, cause race conditions, and will likely be unsupported. Proceed only if you accept that tradeoff. Recap of the limitation • ToolContext is created at the start of a reply pipeline (

_pipeline_reply_task

) and its

all_tools

snapshot is passed into LLM nodes and reused during that reply. •

update_tools()

modifies the agent’s tool list for future pipelines, but it does not mutate the already-constructed ToolContext being used in the current pipeline. High-risk / invasive options (technical details + where to change) 1) Mutate the live ToolContext (patch ToolContext to watch agent.tools) • Idea: Change ToolContext so it does not hold a hard snapshot, but reads from a live source (or can be refreshed) when queried by LLM nodes. • Where to change: ToolContext implementation and any place where

tool_ctx.all_tools

is cached/read (search for ToolContext creation and usages in

_pipeline_reply_task

,

_llm_inference_task

, and nodes that pass tools to the LLM). • Concept: ◦ Replace frozen list with a small wrapper that resolves tools on access; or add a

refresh()

method and cause function tool returns to call it. • Sketch:

# pseudocode conceptual change inside ToolContext
class ToolContext:
    def __init__(self, agent):
        self._agent = agent
        self._snapshot = None
        self._live = True  # new flag

    @property
    def all_tools(self):
        if self._live:
            # resolve from agent every access (or on explicit refresh)
            return self._agent.tools
        return self._snapshot

    def snapshot(self):
        self._snapshot = list(self._agent.tools)
        self._live = False

    def enable_live(self):
        self._live = True

    def refresh(self):
        self._snapshot = list(self._agent.tools)

• How to use: When a function tool calls

update_tools()

, have it call

tool_ctx.refresh()

(you’ll need to surface ToolContext into RunContext or otherwise let tools access it). • Risks: ◦ LLM calls and tool resolution were designed around a stable tool list. Changing it mid-inference can make the LLM’s tool selection inconsistent with the function inputs or change behavior unpredictably. ◦ Concurrency/race conditions if the LLM is mid-token-generation and the callable set changes. ◦ You must wire ToolContext through places currently expecting immutable behavior. 2) Force LLM node to re-request tools between function call return and next inference (re-run llm_node with fresh tools) • Idea: After a tool finishes and returns, intercept the control flow so the next LLM invocation is explicitly called with current agent.tools rather than the cached tools in ToolContext. • Where to change:

_llm_inference_task

/ llm_node invocation site inside the pipeline. Find the logic that performs:

llm_node(chat_ctx, tools, ...)

and alter it so it re-queries agent.tools on each invocation rather than using an outer tools variable. • Sketch:

# inside pipeline where llm_node is invoked:
# original: llm_node(chat_ctx, tools=tool_ctx.all_tools, ...)
# change to:
current_tools = agent.tools  # or tool_ctx.get_latest_tools()
llm_node(chat_ctx, tools=current_tools, ...)

• How to trigger mid-turn update: ◦ Ensure that after a function returns, the pipeline loop re-evaluates

agent.tools

(instead of using a previously-captured variable). • Risks: ◦ Similar to above: the pipeline logic expects a stable tools set for a reply; breaking that assumption may cause LLMs to generate function calls inconsistent with prior state. ◦ Needs careful handling to avoid inconsistent tool metadata passed to the model. 3) Create a mini “sub-pipeline” / synthetic new reply inside the same top-level pipeline (spawn a new pipeline instance) • Idea: When a tool wants immediate new tools, have the function tool itself synthesize a new internal reply invocation (a sub-pipeline) using the updated agent.tools and the current chat history. From the LLM’s perspective, this is equivalent to starting a fresh pipeline; but it is created programmatically inside the function tool. • Where to change/use: implement a helper to run the agent reply runner (the same code path as

_pipeline_reply_task

) but as an internal call that returns the new generated content. You must ensure you don’t deadlock or recursively call the same pipeline routines incorrectly. • Sketch:

# inside function tool after update_tools()
# Build a fresh chat context (copy of current) and call the same code paths used by the pipeline runner.
new_chat_ctx = chat_ctx.copy()
# run the same LLM reply logic but with the updated tools
result = await run_reply_pipeline(agent=self.agent, chat_ctx=new_chat_ctx)
# return or inject the result back into the parent pipeline flow
return result

• Risks: ◦ Complexity and potential recursion if run_reply_pipeline expects to be called only at top-level. ◦ You must ensure any locks, streaming, event hooks are respected; otherwise you can produce duplicated messages, lost events, or race conditions. ◦ Requires intimate knowledge of agent runner internals to call safely. 4) Hold a second LLM instance with fresh ToolContext and bridge calls • Idea: Maintain a lightweight second LLM context that you can create/reset on demand. When tools are updated mid-turn, pass control to the second LLM context which is initialized with the new tools. The parent pipeline must pause and then resume after bridging. • Where to change: orchestration layer that controls LLM node instances and the pipeline loop. • Sketch: ◦ Parent pipeline runs with ToolContext A. ◦ Function requests switch, calls update_tools(), signals orchestrator to start LLM B with ToolContext B. ◦ Orchestrator pauses the current generation, runs LLM B to completion for the next steps, then optionally returns to parent context. • Risks: ◦ Very complex to coordinate streaming and partial outputs. ◦ Likely brittle around cancel/resume and model tokens/state. Recommendation and safe alternatives • If you can modify internals and accept risk: the least invasive and most targeted change is to make the pipeline re-query tools on each LLM invocation (option 2) or to allow ToolContext to be refreshable and expose that refresh to tools (option 1). Both are invasive but more focused than building sub-pipelines or dual LLM orchestration. • If you need production reliability: do not change pipeline invariants. Instead: ◦ Use Agent Handoff (immediate, supported way to create a fresh ToolContext). ◦ Or register a superset and guard selection via strong instructions and tool-side validation. • If you have “repo access” and want me to point to exact files to inspect: ◦ Search for where ToolContext is constructed (look for class or factory named ToolContext and for where it’s instantiated inside

_pipeline_reply_task

). ◦ Search for where LLM node is invoked with tools (look for

llm_node(..., tools=...)

or where

tool_ctx.all_tools

is passed). ◦ Search for

_pipeline_reply_task

,

_llm_inference_task

, and the function tool execution path that returns function results into the pipeline. Those are the hot spots to alter. Concrete next steps I can help with • If you can share the relevant repo files or point me to the exact module names / functions where ToolContext and

_pipeline_reply_task

live in your codebase, I can give exact diffs/patches (pseudo-diffs or concrete code) to implement: ◦ a refreshable ToolContext, or ◦ re-query-before-invoke behavior in the pipeline, or ◦ a safe internal sub-pipeline call pattern. • If you prefer lower-risk, I can provide a fully worked agent-handoff code example tailored to your topic/tool setup (with chat-history transfer, tool construction, and example MCP tool making) so you can get immediate tool effects without touching internals. Which route do you want to take? • I want exact internal patches (I own the repo and accept risk) — I will provide precise file/function names and a proposed patch. • I want a robust, supported approach — show me an agent-handoff + superset hybrid with concrete code. • I want an experimental internal change but need the exact files identified in the LiveKit repo for guidance. Pick one and I’ll proceed with the corresponding level of detail and exact code. , with interactive elements