With gemini 3 flash, I'm getting a lot of tool calls output into the text output stream eg :

startcall:default_api:my_tool_call{}

Thanks for reaching out, Tom. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

What's the best way to filter out a string like this from the llm before it gets to the transcript or tts?

Thanks — I’m responding about filtering out lines like

startcall:default_api:my_tool_call{}

before they reach transcript or TTS. I can’t help with Gemini internals, but here are practical approaches you can apply on your side to remove those strings before they reach your transcript or TTS pipeline: • Preprocess the model output with a simple string filter: ◦ Use a regex to match the pattern and remove or replace it. Example pattern:

startcall:[^:}]+:[^{]+\{[^}]*\}

◦ Run this cleanup immediately after receiving the model text and before storing or sending to TTS. • Normalize and validate output: ◦ Trim whitespace, collapse repeated newlines, and remove empty tokens after filtering to avoid artifacts. • Whitelist allowed characters/phrases: ◦ If you expect only human-readable sentences, strip any token sequences that contain control-like prefixes (

startcall:

,

toolcall:

, etc.) or non-natural punctuation patterns. • Use streaming-safe filtering: ◦ If you process model output incrementally (streaming), buffer a small window (e.g., 256–1024 characters) and apply the regex on the buffered text to avoid cutting tool-call tokens mid-stream before emitting to TTS. • Post-filtering checks: ◦ If TTS expects well-formed sentences, run a quick sanity check (e.g., ensure last emitted char is not mid-token) and only send complete segments. If you want, tell me: • the language/runtime you’re using (Node, Python, etc.) and whether output is streamed or batched, and I’ll give a short code snippet implementing the regex-based filter for that environment., with interactive elements

i'm using python livekit-agents 1.5.2 What's the best place to filter these out? in llm_node? I tried that but it seems to miss most cases. perhaps due to them being split over chunks? __ ``` async for event in Agent.default.llm_node( self, chat_ctx, tools, model_settings ): if isinstance(event, ChatChunk): event = self._strip_tool_call_leaks(event) yield event def _strip_tool_call_leaks(self, event: ChatChunk) -> ChatChunk: content = event.delta.content if content.startswith("startc"): event.delta.content = None return event

Thanks — you’re asking where to best filter tool-call leaks in livekit-agents 1.5.2 and why your

llm_node

approach misses chunked cases. Short answer: keep filtering in

llm_node

, but buffer across chunks and run a regex-based cleanup on the buffered text before emitting. Chunk boundaries can split the marker (e.g., "startc" / "all:..."), so

startswith()

will miss many cases. Suggested pattern (async, streaming-safe): 1. Buffer incoming text fragments until you have a stable segment (small window, e.g., 256–1024 chars) or until a non-text event. 2. Run a regex to remove the tool-call pattern across the buffer. 3. Emit cleaned text chunks (only complete/clean segments) and keep any trailing partial token in the buffer for the next iteration. Example sketch (adapt for your ChatChunk fields):

import re

TOOL_RE = re.compile(r"startcall:[^:}]+:[^{]+\{[^}]*\}")

async for event in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
    if isinstance(event, ChatChunk) and event.delta.content:
        buffer += event.delta.content
        # remove tool-call artifacts across chunk boundaries
        cleaned = TOOL_RE.sub("", buffer)
        # emit clean output and keep trailing partial (last N chars) in buffer
        emit, buffer = cleaned, cleaned[-512:]  # adjust window size as needed
        event.delta.content = emit if emit.strip() else None
        yield event
    else:
        yield event

Alternative places to filter: • If you only need to prevent speech, add a

tts_text_transforms

step so filtering runs just before TTS. • Filter again before you store transcript if you want double-safety. If you want, tell me: • a short real example of how the chunks are split (two consecutive chunks), and I’ll provide a tuned regex and buffer/window sizes for your case. Sources: Function tool definition | LiveKit Documentation | Text and transcriptions | LiveKit Documentation, with interactive elements

will adding the buffer slow down typical processing though ? what's the tradeoffs between using

tts_text_transforms

? ideally id remove this from both the transcript and tts, when users are interracting with my agent in either text or voice modalities, but the voice one is definitely the most disruptive when it outputs this

Thanks — I’ll explain the tradeoffs and give a pragmatic recommendation for removing the artifact from both transcript and TTS. Short answer • Use both: do lightweight buffering+filtering in

llm_node

to keep the transcript clean, and also add a final

tts_text_transforms

filter to guarantee nothing leaks to speech. This minimizes user-facing voice noise while keeping latency impact very small. Tradeoffs and guidance • Latency: a small buffer (e.g., 256–1024 bytes) adds negligible latency for typical streaming (you only delay emission by the time it takes to receive that small window). If you emit as soon as you have clean text (not wait for the whole buffer to fill), perceived latency is minimal. • Robustness:

llm_node

buffering prevents artifacts entering transcripts and downstream logic.

tts_text_transforms

is a final safety net right before speech output and handles any remaining split/edge cases. • Complexity:

llm_node

buffering requires implementing a small stateful buffer and regex cleanup across chunk boundaries.

tts_text_transforms

is simpler to implement but only protects TTS (not transcript/history). • Safety: double-filtering is inexpensive and gives the best UX: transcript + logs are clean and voice never speaks artifacts. Practical implementation notes • Buffer size: 256–1024 chars is usually enough. Smaller means less delay but may increase chance of split tokens; larger increases robustness but slightly more buffering time. • Emission strategy: flush and emit cleaned text as soon as it contains no partial artifact at the end. Keep any trailing partial token in the buffer for the next chunk. • Regex: use a pattern that can match across boundaries like

startcall:[^:}]+:[^{]+\{[^}]*\}

and run it against the buffer before emitting. • TTS transform: apply the same regex-based remover as a final transform on the text stream sent to TTS. Minimal code sketch (adapt fields/names to your ChatChunk structure):

import re

TOOL_RE = re.compile(r"startcall:[^:}]+:[^{]+\{[^}]*\}")
BUFFER_LIMIT = 1024

buffer = ""

async for event in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
    if isinstance(event, ChatChunk) and event.delta.content:
        buffer += event.delta.content
        # remove artifacts across the whole buffer
        cleaned = TOOL_RE.sub("", buffer)
        # keep last BUFFER_LIMIT chars as possible partial
        tail = cleaned[-BUFFER_LIMIT:] if len(cleaned) > BUFFER_LIMIT else cleaned
        emit = cleaned[:-len(tail)] if len(cleaned) > len(tail) else ""
        if emit.strip():
            event.delta.content = emit
            yield event
        # keep trailing part for next iteration
        buffer = tail
    else:
        yield event

Add a simple

tts_text_transforms

that runs the same

TOOL_RE.sub("", text)

on chunks just before TTS. If you want, share a pair of consecutive chunks that currently leak and I’ll tailor the regex and buffer size for that exact split. , with interactive elements

these

startcall:default_api

outputs always appear to happen at the start of messages, does that simplify things at all? or not if it's split across chunks anyway?

Good — since the artifact always appears at the start of messages, filtering becomes simpler but you still must handle chunk splits where the prefix is split across chunks. Quick recommendation • Keep filtering in

llm_node

with a small prefix buffer that only holds enough characters to detect a leading artifact (e.g., 64–256 chars). That minimizes latency. • Also add the same regex as a final

tts_text_transforms

safety net so voice never speaks leaks. Why this works • Because the marker is always at the start, you only need to detect it at segment starts. You can discard or drop content when a buffered prefix matches the pattern, which is cheaper than full-stream buffering. • A tiny buffer prevents missing cases where the "startcall..." is split (e.g., "startc" / "all:..."). Minimal async sketch (adjust field names to your ChatChunk):

import re

TOOL_RE = re.compile(r"^startcall:[^:}]+:[^{]+\{[^}]*\}")

PREFIX_BUF = 256
prefix_buf = ""

async for event in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
    if isinstance(event, ChatChunk) and event.delta.content:
        text = event.delta.content
        # if we have no prefix yet, accumulate a small prefix window
        if len(prefix_buf) < PREFIX_BUF:
            need = PREFIX_BUF - len(prefix_buf)
            prefix_buf += text[:need]
        # check if prefix_buf indicates a leading tool-call
        if TOOL_RE.match(prefix_buf):
            # drop leading artifact from the combined stream
            # remove matching prefix from text+prefix_buf
            combined = (prefix_buf + text)
            cleaned = TOOL_RE.sub("", combined, count=1)
            # emit cleaned remainder; reset prefix buffer to next trailing slice
            event.delta.content = cleaned
            prefix_buf = cleaned[-PREFIX_BUF:]
        else:
            # normal case: emit text and keep last PREFIX_BUF chars as prefix_buf
            event.delta.content = text
            prefix_buf = (prefix_buf + text)[-PREFIX_BUF:]
        yield event
    else:
        yield event

Add the same

TOOL_RE.sub("", text)

as a

tts_text_transforms

function to remove any remaining artifacts before TTS. If you share two consecutive chunk strings where the marker is split, I’ll tune the prefix size and the regex for your exact case. , with interactive elements