https://livekit.io logo
Join Slack
Powered by
I have a problem that my voicebot built with livek...
# ask-ai
f
I have a problem that my voicebot built with livekit is speaking the tool call input out loud to the user. Things like: role: "assistant" content: "{"recipient_name":"functions.disturbance_info","parameters":{}} Selge, räägime eesti keeles, aitäh selgitamast. Üks hetk, kontrollin esmalt, kas on üldisemaid rikkeid. Could you explain what could be the cause of this behaviour
t
Thanks for reaching out, Heidi. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I understand you're seeing raw tool-call content (JSON + role labels) being spoken aloud by your LiveKit voicebot. I’ll explain the likely causes and give practical steps you can take to stop that from happening. If you want, I can walk through code examples after you tell me which SDK/language and TTS pipeline you’re using. What’s happening (root cause) • The LLM/agent is including internal tool-call data (e.g., role: "assistant", content: "{...}" or serialized JSON) in its output. • That LLM output is then passed unchanged to your text-to-speech (TTS) pipeline, so the TTS reads the internal tool call text verbatim. • This typically happens when the agent does not separate internal tool execution details from the user-facing response or when there’s no post-processing step to remove or transform tool-call artifacts before TTS. Common triggers • The LLM is returning chain-of-thought or raw function-call traces in its assistant messages. • Tool responses or function-call arguments are appended to the assistant message instead of being consumed or summarized. • Missing filters or markers that identify “internal” vs “user-facing” text, so TTS receives everything. How to fix it (practical steps) 1. Post-process LLM output before sending to TTS ◦ Remove or redact any text that looks like serialized JSON, function-call metadata, or role labels. ◦ Example filters: strip lines that start with 
role:
, remove top-level JSON blocks, or use a regex to remove 
{ ... }
tool payloads. 1. Make the agent produce a clear user-facing message ◦ Configure the agent or prompt so it returns a final user-facing reply that excludes tool-call details. ◦ Ask the LLM to summarize tool responses into plain language (e.g., “Found disturbance info: [summary]”) and only send that summary to TTS. 1. Use structured channels / message types (if available) ◦ If your pipeline or SDK supports message metadata or separate channels (e.g., 
tool
, 
assistant
, 
system
), route tool outputs to a non-TTS channel and only route 
assistant.user_message
content to TTS. 1. Detect and strip internal tokens or chain-of-thought ◦ Some models output chain-of-thought or special markers. Detect those markers and remove those segments before TTS. 1. Sanity-check tool outputs before synthesis ◦ If a tool returns JSON, convert it to a human-friendly sentence first (e.g., “I checked disturbance info — no problems found.”) and send that to TTS. ◦ Never send raw serialized objects to your TTS engine. Questions to help me give exact code or config • Which LiveKit SDK / language are you using (JS/TS, Python, other)? • Where in your pipeline do you call TTS (client-side, server-side)? • Do you receive separate message roles (e.g., 
tool
, 
assistant
) from your LLM/agent or is everything combined in a single string? • Do you want a quick code example for filtering/parsing and sending only the cleaned text to TTS? If you tell me your SDK/language and whether TTS is run server- or client-side, I’ll provide a short example to filter out the tool-call JSON and ensure only user-facing text is spoken.
f
I am using Python. I call TTS on server side The model is GPT 5.2.
t
Thanks — I see you’re using Python, calling TTS server-side, and the model is GPT 5.2. You’re hearing raw tool-call JSON because the model’s assistant output (which includes tool call / function-call details) is being passed straight to your TTS engine. Below I’ll explain why that happens and give a practical Python approach to prevent it. Why this happens • The agent/LLM returns both user-facing text and internal tool-call information (role labels, serialized JSON parameters). • If you forward the LLM output directly to TTS, the TTS will speak any text it receives — including those internal tool-call strings. • The correct flow is to execute/consume tool outputs internally, then produce a clean, human-readable reply that TTS synthesizes. Recommended fix (server-side Python) • Intercept the LLM output stream before it reaches your TTS node. • Detect chunks that represent tool calls or contain serialized JSON/role metadata. • Skip or transform those chunks so only natural language responses go to TTS. • One way to do this is to override the 
llm_node
in your Agent and filter streaming chunks that contain tool-call info. Example pattern (conceptual) 1. Override 
llm_node
in your Agent subclass. 2. Stream the default LLM output. 3. If a chunk contains tool-call metadata (for example a 
tool_calls
field, role labels, or a JSON payload), do not yield it to the TTS stage. 4. Optionally, convert tool results into a short natural-language summary and yield that instead. Minimal illustrative code (adapt to your environment)
Copy code
from livekit.agents import Agent, ChatContext, FunctionTool, ModelSettings
from typing import AsyncIterable

class Assistant(Agent):
    async def llm_node(
        self,
        chat_ctx: ChatContext,
        tools: list[FunctionTool],
        model_settings: ModelSettings,
    ) -> AsyncIterable:
        # Stream the default LLM output
        async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
            # Detect tool calls or serialized JSON in the chunk and skip them
            if hasattr(chunk, "tool_calls") and chunk.tool_calls:
                continue  # do not send tool-call chunks to TTS

            # Optionally: detect large JSON blocks or role lines and skip/clean them
            text = getattr(chunk, "delta", None) and getattr(chunk.delta, "content", None)
            if text:
                # basic JSON / role-line guard (tune regex to your needs)
                if text.strip().startswith("{") or text.strip().startswith("role:"):
                    continue
                # If clean, yield chunk to downstream (TTS) node
                yield chunk
Additional recommendations • Instead of simply skipping, you can generate a compact summary of tool outputs (e.g., “Checked disturbance info — no issues found.”) and send that to TTS so users get meaningful feedback. • If your agent or SDK supports separate channels/metadata for tool outputs, ensure you route those away from TTS and only route assistant/user-facing messages to the speech pipeline. • Add logging for skipped chunks so you can verify filtering is working and troubleshoot edge cases. If you’d like, I can: • Provide a more complete code example tailored to your exact LiveKit SDK version and TTS library. • Show a small routine to convert tool JSON into a human-readable summary before sending it to TTS. • Suggest a regex or parser tuned to the specific JSON/tool-call format you’re seeing. Which would you prefer? If you paste a short real example of the LLM chunk structure (the raw message object you receive before TTS), I’ll produce an exact filter/snippet you can drop into your agent. Sources: Tool definition and use | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation
2 Views
Open in Slack
PreviousNext
Powered by