https://livekit.io logo
Join Slack
Powered by
I have a tool call that my agent is going to make ...
# ask-ai
r
I have a tool call that my agent is going to make frequently, but will never care about the result because it's simply to extract and record context data while the conversation progresses. Currently it's requiring two hits to the LLM (one for the tool call and one after the tool call completes to generate the next textual response to the user). How can I turn this into a single LLM hit to both trigger the storing of data, but also include what to say to the user next?
✅ 1
t
Thanks for reaching out, James. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I understand you want one LLM hit that both triggers a background tool call (to record context) and produces the agent’s next message, to avoid two model invocations. Short answer: you can’t force a single LLM invocation to both execute a tool and then produce the final user-facing reply. LiveKit uses a two-step sequence: the model chooses to call a tool, the tool runs, and then the model is invoked again (with the tool result) to generate the reply. [Sequential pipeline; ReAct pattern] What you can do instead (practical options): • Mark the tool as silent. ◦ Set the tool to return no value or configure 
silent: true
in the Agent Builder so the agent does not generate a follow-up response specifically about the tool result. This avoids an extra visible interaction and keeps the agent focused on the conversation. [Function tool definition; Agent Builder] • Have the LLM include the next user message in its tool call intent (embedding the reply in the tool payload). ◦ The model still initiates a tool call, but you can design the tool’s input to include the text the agent should say next. The tool executes and records data; your system then returns that pre-composed reply as the agent response. Note: LiveKit still performs the tool-execute → response pattern, so this does not change that there are two internal steps, but it saves the second model generation. • Optimize for latency and cost: ◦ Use the smallest/lower-cost model capable of the decision to call the tool, or cache model decisions where possible, so the extra step is cheaper/faster. Also make the tool execution extremely fast (local DB write, in-memory queue) so the overall latency is minimal. If you’d like, tell me which agent builder you’re using (HTTP client tool, custom tool, or function tool) and whether you prefer the reply to be pre-composed by the LLM or generated after execution — I’ll provide the exact configuration or example payload you can use. Sources: Sequential Pipeline Architecture for Voice Agents | LiveKit | The ReAct Pattern for Voice Agents and How AI Agents Think, Act, and Respon... | Large language models (LLM) overview | LiveKit Documentation | Function tool definition | LiveKit Documentation | Agent Builder | LiveKit Documentation, with interactive elements
r
Using the python sdk and this being a function_tool defined, what does an implementation of this look like?
t
Thanks — I’m responding to your request for a Python SDK example of a 
function_tool
that records context silently (so the agent doesn’t need a second LLM call to acknowledge the tool). Short answer: return 
None
(or omit a return) from the 
function_tool
to make it silent. Below is a minimal example showing two options: • Silent tool that only stores context (no LLM follow-up). • Tool that also returns a pre-composed agent reply (the tool runs, then you can return that text as the agent response).
Copy code
from livekit.agents import Agent, function_tool, RunContext

class MyAgent(Agent):
    # Option A: Silent recorder — records context and returns None (no LLM reply)
    @function_tool()
    async def record_context_silent(self, context: RunContext, key: str, value: str) -> None:
        """Store context metadata silently; agent will NOT generate a follow-up reply."""
        # example: store in session metadata or chat context
        context.session.chat_ctx.set_metadata(key, value)
        # returning None (or not returning) makes this tool silent

    # Option B: Recorder that also supplies the next reply text
    @function_tool()
    async def record_context_with_reply(self, context: RunContext, key: str, value: str) -> dict:
        """Store context and provide the pre-composed agent reply."""
        context.session.chat_ctx.set_metadata(key, value)
        # Compose the reply here so your system can emit it without a second model call.
        # Return a structure the agent runner can use; e.g.:
        return {"agent_reply": f"Thanks — I've saved your {key}."}
Notes and recommendations: • Option A: Best when you want the tool to run silently and let the agent continue without generating a tool-result-driven reply. • Option B: LiveKit still calls the tool then generates a reply step, but by returning the pre-composed reply from the tool you can avoid invoking the LLM again to craft that message (your runner must be wired to use the returned 
agent_reply
as the outgoing message). • Make the tool execution fast (in-memory or quick DB write) to keep overall latency low. If you want, tell me which pattern you prefer (A or B) and I’ll provide a concrete example of wiring the agent runner to use the tool-returned reply in Option B. Sources: Function tool definition | LiveKit Documentation | Agent Builder | LiveKit Documentation | Collect structured data with LiveKit Agents | LiveKit | The Handoff Pattern for Voice Agents That Replaces IVR Menus | LiveKit, with interactive elements
r
What is the 
StopResponse
helper for and could it be helpful here?
t
Acknowledging your question about `StopResponse`: it’s a helper to stop the agent from generating a reply after a tool or lifecycle hook runs. What it does • Raising 
StopResponse
inside a tool, hook, or pipeline node tells the runner to suppress the agent’s follow-up response. Use it when an action should have no verbal acknowledgement. How it helps your case • It will not merge tool execution and LLM generation into one call. LiveKit still sequences tool execution and response generation. • It is useful to avoid an extra user-facing reply after your context-recording tool runs — similar effect to a silent tool. Python example
Copy code
from livekit.agents import function_tool, StopResponse, RunContext

class MyAgent:
    @function_tool()
    async def record_context_and_stop(self, context: RunContext, key: str, value: str) -> None:
        # store context
        context.session.chat_ctx.set_metadata(key, value)
        # prevent the agent from generating a reply
        raise StopResponse()
When to use which approach • Use a silent 
function_tool
(return 
None
) when you simply want the tool to run without causing any reply. • Use 
StopResponse
when you need conditional suppression inside hooks or more complex logic. • If you want the agent to say something but avoid a second LLM call, have the tool return a pre-composed reply (and wire your runner to use it) — 
StopResponse
does not create or return such text. Tell me which option you prefer (silent tool, StopResponse, or tool-returned reply) and I’ll give the exact wiring example for the Python runner. Sources: Pipeline nodes and hooks | LiveKit Documentation | Agent Builder | LiveKit Documentation, with interactive elements
r
Can we combine the silent tool approach with adding a function variable for what to say next, so that we can use 
StopResponse
and 
session.say
together to combine the tool and say functionality into a single pass?
t
Yes — you can combine a silent 
function_tool
with 
session.say()
and 
StopResponse
to record context and speak a pre-defined reply without an extra LLM-driven response. That achieves the behavior you described: the LLM invokes the tool, the tool records state and triggers speech, and no follow-up generation is produced. Below is a compact Python pattern you can use. 1) function tool that records context and calls session.say()
Copy code
from livekit.agents import Agent, function_tool, RunContext
from livekit.agents.llm import StopResponse

class MyAgent(Agent):
    @function_tool()
    async def record_and_speak(self, context: RunContext, next_message: str) -> None:
        # 1) persist context (session metadata / DB / state)
        context.session.chat_ctx.set_metadata("last_note", next_message)

        # 2) schedule outgoing speech immediately (bypasses LLM reply)
        # `allow_interruptions` optional depending on your UX needs
        await context.session.say(next_message, allow_interruptions=True)

        # 3) ensure tool is silent (return None) — the runner won't surface a tool result
        return None
2) lifecycle hook to suppress the LLM reply after the tool runs
Copy code
async def on_user_turn_completed(self, ctx: RunContext, new_message) -> None:
    # Detect that your tool ran and scheduled speech (use a flag or inspect session/chat_ctx)
    last_note = ctx.session.chat_ctx.get_metadata("last_note")
    if last_note:
        # prevent the agent from generating a follow-up LLM response
        raise StopResponse()
Notes and recommendations • The model still makes a tool-choice call and the tool executes; you are not merging two model invocations into a single raw LLM API call. Instead, you avoid the second model-generated reply by having the tool emit speech directly and stopping the agent from generating text. • Use a lightweight, deterministic 
next_message
value (or precompose it) so the tool can call 
session.say()
without needing further LLM reasoning. • Use a clear flag (session metadata or an agent attribute) to signal the lifecycle hook that speech was scheduled, then raise 
StopResponse()
to suppress the follow-up. • Ensure the tool action (DB write, metadata update, say) is fast to keep latency low. If you want, I can provide a complete runnable example showing how to set the metadata flag inside the tool, clear it after use, and how to wire this into your agent runner. Which runner/version of the Python SDK are you using? Sources: Function tool definition | LiveKit Documentation | Audio customization | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | The ReAct Pattern for Voice Agents and How AI Agents Think, Act, and Respon..., with interactive elements
r
and what would we need in order to flip the 
last_note
back such that the LLM will respond when the user talks again?
t
Good question — you just need to clear the flag you set when the tool ran (e.g., 
last_note
) once the tool-scheduled speech has been delivered, so subsequent user turns are treated normally. A simple, reliable pattern: • Set the flag in the tool when you call 
session.say()
(you already do this). • In the lifecycle hook that runs after the agent’s outgoing speech finishes or at the start of the next user turn, clear that flag. • Only raise 
StopResponse()
if the flag is present. When cleared, the hook will not raise and the LLM will generate a reply as usual. Minimal example (continuing prior code):
Copy code
# inside the function tool
await context.session.say(next_message, allow_interruptions=True)
context.session.chat_ctx.set_metadata("last_note", "pending")

# lifecycle hook to suppress reply immediately after tool
async def on_user_turn_completed(self, ctx: RunContext, new_message) -> None:
    flag = ctx.session.chat_ctx.get_metadata("last_note")
    if flag == "pending":
        # clear the flag so next user turn will allow LLM responses
        ctx.session.chat_ctx.set_metadata("last_note", None)
        raise StopResponse()

# optional: if you have an event that fires after speech finished, clear flag there instead
async def on_speech_finished(self, ctx: RunContext) -> None:
    ctx.session.chat_ctx.set_metadata("last_note", None)
Recommendations • Use a deterministic value (e.g., "pending") and clear to 
None
or delete the key. • Clear the flag as soon as the speak action is complete (either in a speech-finished hook or the first lifecycle hook after the speak call) to avoid accidentally suppressing the next user turn. • If interruptions are allowed, account for cases where the user speaks during the agent’s speech — you may want to clear the flag immediately on interruption so the LLM can respond. If you tell me which lifecycle events your runner exposes (e.g., 
on_speech_finished
, 
on_user_turn_started
, or only 
on_user_turn_completed
), I can tailor the exact hook placement. , with interactive elements
3 Views
Open in Slack
PreviousNext
Powered by