Thanks — that sounds frustrating. I’m responding to your issue about internal system messages (like chain-of-thought tags or tool-call markers) leaking into chat responses and spoken output, and I’ll share practical mitigation steps you can try. Short answer • Intercept the model’s streaming output and filter/replace unwanted tokens before it reaches the user or the TTS engine. Overriding the agent’s

llm_node

to process chunks as they stream is an effective approach. What you can do (practical options) • Override the

llm_node

(or equivalent output hook) to intercept streaming LLM output: ◦ Inspect each streaming chunk and remove or replace model-internal tokens such as

<tool_call>

,

</tool_call>

, or common chain-of-thought markers (e.g., “Having”, internal reasoning phrases). ◦ Optionally replace them with short natural transitions (for TTS) like “Okay — here’s the answer:” so spoken audio sounds natural. • Apply targeted regex/text filters: ◦ Remove known tags/patterns rather than blanket sanitization to avoid cutting legitimate content. ◦ Example patterns:

<tool_call>.*?</tool_call>

(strip), or single-token replacements for specific model markers. • Use a two-stage pipeline for spoken output: ◦ Stage 1: Produce the final textual response visible to the user (with internal messages removed). ◦ Stage 2: Feed only that sanitized text to the TTS engine. • Add post-processing safeguards: ◦ Run a short profanity/format verification pass to catch any leftover markup before display/TTS. ◦ If possible, validate that the cleaned output still satisfies any content constraints (completeness, clarity). • Prefer model-level settings when available: ◦ If your model or system supports controlling chain-of-thought or tool-output verbosity, reduce internal reasoning verbosity at generation time so less internal content is produced. Things to watch out for • Overzealous filtering can remove legitimate text. Prefer specific patterns or tag lists over broad heuristics. • If you rely on streaming and need low-latency TTS, ensure your filter runs efficiently (chunk-based processing) so you don’t introduce notable lag. • Keep a small log of filtered snippets during testing so you can tune rules without losing visibility into what the model tried to output. If you want, I can help with any of the following next steps • Suggest concrete regex examples or a sample chunk-processing function to remove tags you’re seeing. • Review a short sample of the model output you’re trying to filter (paste a representative snippet) and propose exact replacements. • Advise on how to integrate the filter into your streaming pipeline for minimal latency. Which would be most helpful to you right now — a sample code snippet for chunk filtering, or help tuning specific patterns you’re seeing in your outputs? Sources: LLM Output Replacement | LiveKit Documentation | Modifying LLM output before TTS | LiveKit Documentation