Hey, how do you determine which chunk is thinking chunk that the tts model doesn't want to read and which it will read?

Thanks for reaching out, Alexey. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

first i can't access llm_node because i am using realtime model second Livekit does determine which chunk to skip and which to read. but i can't see where it is being determined. i want to isolate thinking from being sent as raw text and label it somehow. here is the google realtime model configuration i've set up for myself: google_realtime_model = google.realtime.RealtimeModel( api_key=settings.GEMINI_API_KEY, api_version="v1beta", model=agent.model, voice=agent.voice_model, language=( # NOTE: Hebrew is not supported by Google Realtime API yet agent.language if agent.language != "he-IL" else NOT_GIVEN ), temperature=agent.temperature, thinking_config=ThinkingConfig( include_thoughts=False, thinking_budget=-1, thinking_level=ThinkingLevel.LOW, ), tool_behavior=Behavior.NON_BLOCKING, tool_response_scheduling=FunctionResponseScheduling.WHEN_IDLE )

Thanks — I hear that you’re using the Google realtime model, you can’t access

llm_node

, and you believe LiveKit is already deciding which chunks to skip but you can’t find where that decision happens. I can help you isolate and label thinking text without

llm_node

. A couple quick clarifying questions first: 1. Which LiveKit SDK/environment are you running (Python server agent, JS client, or hosted LiveKit agent)? 2. Do you want the model to emit labeled thinking text (so you can handle it), or do you want thinking to never be emitted at all and simply removed from all outputs? Once you confirm, I’ll give the minimal change you can make (config or prompt pattern) and a short code snippet or event-listener suggestion tailored to your environment. Sources: Pipeline nodes and hooks | LiveKit docs | LLM Output Replacement | LiveKit docs | Modifying LLM output before TTS | LiveKit docs | Gemini Live API plugin | LiveKit docs | OpenAI Realtime API plugin guide | LiveKit docs

python SDK. i want to somehow mark thinking, determine which chunk is a thinking process or a transcription text, the transcription node seems to be sending it as basic str.

Thanks — since you’re on the Python SDK and

transcription_node

currently yields plain strings, here are concise, practical options to label/isolate thinking chunks given realtime model constraints. Quick facts I’m using: • You can’t use

llm_node

for realtime audio mode. •

thinking_config.include_thoughts

controls whether the model emits internal thoughts as events. • If the model emits thinking as text, it will arrive as strings unless you add structure yourself. Options (pick one): 1) Inspect thoughts by enabling them, then label and drop them • Temporarily set

include_thoughts=True

to observe how the model emits thoughts. • In your

transcription_node

, detect patterns the model uses for thoughts (e.g.,

<tool_call>

,

Having

, or a bracket prefix) and tag/skip those strings. Example pattern-based

transcription_node

(Python):

import re
from typing import AsyncIterable

THINK_PATTERNS = [
    re.compile(r"^<tool_call>"),     # adjust to actual markers
    re.compile(r"^\[THINK:"),        # if you instruct the model to label thoughts
    re.compile(r"^Having\b", re.I),
]

async def transcription_node(self, text: AsyncIterable[str], model_settings=None):
    async for chunk in text:
        # chunk is a plain str from realtime model
        if any(p.match(chunk) for p in THINK_PATTERNS):
            # labeled as thinking — handle separately or drop
            yield {"type": "thinking", "text": chunk}
        else:
            yield {"type": "transcript", "text": chunk}

Note: you convert the raw string into small dicts so downstream code can distinguish types. 2) Instruct the model to emit labeled thoughts (recommended if markers are inconsistent) • Add instruction/preamble to agent prompts: ask the model to wrap internal thoughts in a clear token like

[THINK: ...]

and not to include that token in spoken output. • Keep

include_thoughts=True

while testing so you can confirm the model follows the format, then parse with the same

transcription_node

above. Example instruction snippet to include in system prompt: "You may produce internal reasoning. When you do, wrap it in

[THINK: ...]

. Do not include

[THINK:]

in the spoken final response." 3) Force text-only realtime + local TTS (if you want full control) • Configure the realtime model to produce only text: ◦ set

modalities=["text"]

on the RealtimeModel ◦ keep

thinking_config.include_thoughts=False

or True while testing • Then you can run `llm_node`/`tts_node` locally and have full control to label and drop thinking before sending to TTS. Minimal change to your config:

google_realtime_model = google.realtime.RealtimeModel(
    api_key=settings.GEMINI_API_KEY,
    api_version="v1beta",
    model=agent.model,
    voice=agent.voice_model,
    language=agent.language,
    temperature=agent.temperature,
    thinking_config=ThinkingConfig(include_thoughts=True),  # enable for inspection
    modalities=["text"],  # optional: use if you want local TTS control
)

Which approach do you want to try first — (A) pattern-detection in

transcription_node

(I can tighten patterns for your model), (B) prompt-instruct the model to label thoughts, or (C) switch to text-only + local TTS so you can use `llm_node`/`tts_node`?