https://livekit.io logo
Join Slack
Powered by
When using gemini 3 models the agent response is c...
# ask-ai
w
When using gemini 3 models the agent response is coming a object of thought and response
t
Thanks for reaching out, Test. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m seeing you get an object containing both "thought" and "response" when using Gemini 3. I can help you remove the thoughts and return only the final text. Quick checklist and next steps: 1) Disable thinking output • Set the model’s thinking config to exclude thoughts:
Copy code
thinking_config=types.ThinkingConfig(include_thoughts=False)
Use this when creating the `RealtimeModel`/agent session so only the final response is returned. 2) Confirm model/plugin support • Thinking mode is supported only on specific Gemini preview models. If you’re using a model that doesn’t support the config, thoughts may still appear. Tell me which model string you’re using (e.g., 
gemini-...
) so I can confirm. 3) Check local processing • If you have any custom 
llm_node
or response-postprocessing that reconstructs the output, that code can re-introduce a 
thought
field. Share a short sample of the raw model response (redact secrets) or paste the code that handles incoming messages. Please reply with: • The exact model name you’re using • A short example of the returned object (or the code that handles the response) I’ll use that to give the exact config or code change you need. Sources: Gemini Live API plugin | LiveKit docs | Modifying LLM output before TTS | LiveKit docs
w
thinking_config=types.ThinkingConfig(include_thoughts=False) this did not worked
t
Thanks — I’m sorry that setting didn’t take effect. I can help figure out why. Could you share the following so I can diagnose it quickly? • The exact model ID string you’re using (copy-paste). • The code snippet where you create the `RealtimeModel`/agent session (redact any keys). • A short sample of the raw response object you’re receiving (redact sensitive data). Also confirm: • You’re importing 
types
from 
from google.genai import types
. • You are using 
google.realtime.RealtimeModel
(not 
google.LLM
). With those I’ll tell you whether the model supports 
include_thoughts=False
or point to the exact code change needed. Sources: Gemini Live API plugin | LiveKit docs
w
google.LLM( model="gemini-3-flash-preview", api_key=os.getenv("GOOGLE_API_KEY"), vertexai=False, thinking_config=thinking_config, )
t
Thanks — I can see the problem from your snippet. What’s happening • You’re using 
google.LLM
with 
model="gemini-3-flash-preview"
. The 
thinking_config.include_thoughts=False
option is not supported for that combination. • Thinking mode suppression is supported only when using the Live API realtime plugin 
google.realtime.RealtimeModel
with specific native-audio preview models (for example: 
gemini-2.5-flash-native-audio-preview-09-2025
). For 
gemini-3-flash-preview
with 
google.LLM
, thoughts will still be returned. Two ways to proceed • Switch to a supported realtime model and plugin so you can disable thoughts:
Copy code
from google.genai import types
from livekit.plugins import google

session = AgentSession(
  llm=google.realtime.RealtimeModel(
    model="gemini-2.5-flash-native-audio-preview-09-2025",
    thinking_config=types.ThinkingConfig(include_thoughts=False),
  )
)
• Keep using 
gemini-3-flash-preview
and filter out the 
thought
field in your post-processing code (remove or ignore the 
thought
key before using/sending the response). Tell me which option you prefer and I’ll: • Give the exact code to switch to 
RealtimeModel
, or • Provide a small snippet to strip out the 
thought
field from your current responses.
w
• Keep using 
gemini-3-flash-preview
and filter out the 
thought
field in your post-processing code (remove or ignore the 
thought
key before using/sending the response). code for this
t
Thanks — here’s a concise helper you can drop into your post-processing to remove the 
thought
field and return the final text when using 
gemini-3-flash-preview
.
Copy code
def clean_gemini_response(resp):
    """
    Accepts the raw response (dict or string) and returns a dict with:
      - 'text': final response text (if found)
      - other metadata except 'thought'
    """
    if isinstance(resp, str):
        return {"text": resp}

    if not isinstance(resp, dict):
        return {"text": str(resp)}

    # Remove 'thought' if present
    cleaned = {k: v for k, v in resp.items() if k != "thought"}

    # Prefer explicit fields, fall back to first string value
    final_text = cleaned.get("response") or cleaned.get("text")
    if final_text is None:
        for v in cleaned.values():
            if isinstance(v, str):
                final_text = v
                break

    if final_text is not None:
        # Return a compact object with text plus remaining metadata
        meta = {k: v for k, v in cleaned.items() if k not in ("response", "text")}
        return {"text": final_text, **meta}

    return cleaned
Use it immediately after you receive the model output (before sending to TTS or downstream). If you need a version that handles streaming chunks, tell me how you receive the stream and I’ll provide that snippet.
w
where to call this function
t
Call 
clean_gemini_response
immediately after you receive the raw model output and before any downstream use (TTS, UI, agent logic, storage). Below are two short examples showing common places to insert it. 1) Synchronous generate call
Copy code
# where you call the LLM and get a response dict
raw = google.LLM(...).generate(prompt=prompt)
clean = clean_gemini_response(raw)
final_text = clean["text"]

# then send to TTS / UI / further processing
tts.play(final_text)
2) Response handler in an agent or message loop
Copy code
def handle_model_output(raw_resp):
    clean = clean_gemini_response(raw_resp)
    text = clean["text"]
    # use text only (or pass remaining metadata as needed)
    send_to_client(text)

# Example usage inside your event loop
for raw_chunk in model_stream:
    # raw_chunk may be partial; you can buffer then call when complete
    handle_model_output(raw_chunk)
If you share the exact call you use to get model responses (sync 
generate
, async, or streaming), I’ll show the exact line to insert the function.
w
should i override the llm_node in agent?
t
Good question — overriding the 
llm_node
is optional. Use it if you want a single centralized place to strip 
thought
from every model response. If you only need the filter in one spot, calling the helper right after the generate/response is simpler. Below are two concise options so you can pick one. Option A — Do NOT override llm_node (simpler) • Call 
clean_gemini_response
immediately after each place you call the LLM (sync generate or in your response handler).
Copy code
raw = google.LLM(...).generate(prompt=prompt)
clean = clean_gemini_response(raw)
final_text = clean["text"]
# proceed with final_text
Option B — Override / wrap the llm_node (centralized, recommended for agents) • Create a thin wrapper node that calls the original node and filters the output. Register this node in your AgentSession so all agent calls go through the filter. Example pattern (pseudo-code — adapt to your agent framework):
Copy code
class FilteredLLMNode(OriginalLLMNode):
    def call(self, *args, **kwargs):
        raw = super().call(*args, **kwargs)
        return clean_gemini_response(raw)

# when building agent/session:
agent_session.register_node("llm", FilteredLLMNode(...))
When to choose: • Small project / few calls: Option A. • Agent with many flows or streaming: Option B for consistency. If you want, paste your agent setup code and I’ll provide the exact wrapper implementation for your framework.
w
i am using Agents with llm node and not using generate directly
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
w
clean_gemini_response where to call this method in a agent flow
Open in Slack
PreviousNext
Powered by