Am I able to edit thinking config for gemini models being used in speech inference with livekit sdk?

Thanks for reaching out, Tigran. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

short python example would be greate. Right now my code looks like: def create_google_llm_with_config(converse_type: str | None = None) -> google.LLM: """ Create a Google LLM instance with optimized settings for timeout prevention. PERFORMANCE OPTIMIZATION: - Sliding window (conversation history limiting) is handled in llm_node - Prevents timeouts after 15+ minutes of conversation - Reduces API costs and latency dramatically This configuration includes: - Latest Gemini model for best performance - Optimized parameters for voice chat scenarios - Special configuration for singlePrompt converseType Note: Conversation history limiting happens in llm_node before the LLM is called, so all LLM providers (Google, Together, vLLM, OpenAI) benefit from it automatically. Args: converse_type: The conversation type from case template settings Returns: google.LLM: Configured Google LLM instance """ # Use different model for singlePrompt (penalties not supported by 2.5-flash) if converse_type == "singlePrompt" or converse_type == "singlePrompt2": return google.LLM( model="gemini-3-flash-preview", # Use 3-flash for singlePrompt temperature=0.7, # Timeout prevention parameters top_p=0.9, # Nucleus sampling for more focused responses top_k=40, # Top-k sampling to limit vocabulary and speed up generation # Use LOW thinking level for faster responses in singlePrompt mode thinking_config=ThinkingConfig(thinking_level="LOW"), ) else: return google.LLM( model="gemini-3-flash-preview", # Latest model for best performance temperature=0.7, # Timeout prevention parameters top_p=0.9, # Nucleus sampling for more focused responses top_k=40, # Top-k sampling to limit vocabulary and speed up generation # Safety filters OFF for medical/therapy simulations # Allows patients to discuss sensitive health topics freely safety_settings=[ types.SafetySetting( category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH, threshold=types.HarmBlockThreshold.OFF, ), types.SafetySetting( category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold=types.HarmBlockThreshold.OFF, ), types.SafetySetting( category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold=types.HarmBlockThreshold.OFF, ), types.SafetySetting( category=types.HarmCategory.HARM_CATEGORY_HARASSMENT, threshold=types.HarmBlockThreshold.OFF, ), ], )

Thanks — responding to your request for a short Python example showing how to add

thinking_config

to your existing factory function. Here's a concise example that follows your structure and shows

ThinkingConfig

usage for both branches:

from google.genai import types
from livekit.plugins import google
from google.genai.types import ThinkingConfig

def create_google_llm_with_config(converse_type: str | None = None) -> google.LLM:
    """
    Create a Google LLM instance with optimized settings for timeout prevention.
    """
    base_kwargs = dict(
        model="gemini-3-flash-preview",
        temperature=0.7,
        top_p=0.9,
        top_k=40,
    )

    if converse_type in ("singlePrompt", "singlePrompt2"):
        # Use LOW thinking level for faster responses in singlePrompt mode
        base_kwargs.update(
            thinking_config=ThinkingConfig(thinking_level="LOW")
        )
        # Use a different model if you need penalties unsupported by 2.5-flash
        base_kwargs["model"] = "gemini-3-flash-preview"
        return google.LLM(**base_kwargs)
    else:
        # Example: disable safety filters (as in your original code)
        safety_settings = [
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_HARASSMENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
        ]

        base_kwargs.update(
            thinking_config=ThinkingConfig(thinking_level="MEDIUM"),
            safety_settings=safety_settings,
        )
        return google.LLM(**base_kwargs)

If you want, I can: • show how to set

include_thoughts=False

, or • provide an example using a Gemini model that specifically supports thinking (e.g., the 2.5 flash native audio preview) — which would require swapping the

model

name. Which would you prefer?

livekit_agent | 2025-12-19 200551,752 - INFO app.liveKit.utils.llm_factory - [case_id:a0d9adc1-278e-4d29-96d9-8e3a38d63729] 2025-12-19 200551,752 - app.liveKit.utils.llm_factory - INFO - Using this LLM provider: google_default for converse type: singlePrompt2 {"pid": 62, "job_id": "AJ_AtGnKEwZHXKd"} livekit_agent | [case_id:a0d9adc1-278e-4d29-96d9-8e3a38d63729] 2025-12-19 200551,754 - my-worker - ERROR - Error in entrypoint: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden livekit_agent | Traceback (most recent call last): livekit_agent | File "/llm-service/app/liveKit/agent.py", line 201, in entrypoint livekit_agent | agent = await run_voice_pipeline_agent(ctx, participant) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/agent.py", line 324, in run_voice_pipeline_agent livekit_agent | agent = await setup_voice_pipeline_agent( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/agent_utils.py", line 264, in setup_voice_pipeline_agent livekit_agent | llm=get_llm_by_provider(llm_provider, converse_type), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 504, in get_llm_by_provider livekit_agent | return factory_fn(converse_type) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 415, in <lambda> livekit_agent | "google_default": lambda converse_type: create_google_llm_with_config( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 103, in create_google_llm_with_config livekit_agent | thinking_config=ThinkingConfig(thinking_level="LOW"), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 250, in init livekit_agent | validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | pydantic_core._pydantic_core.ValidationError: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden livekit_agent | 2025-12-19 200551,754 - ERROR my-worker - [case_id:a0d9adc1-278e-4d29-96d9-8e3a38d63729] 2025-12-19 200551,754 - my-worker - ERROR - Error in entrypoint: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden livekit_agent | Traceback (most recent call last): livekit_agent | File "/llm-service/app/liveKit/agent.py", line 201, in entrypoint livekit_agent | agent = await run_voice_pipeline_agent(ctx, participant) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/agent.py", line 324, in run_voice_pipeline_agent livekit_agent | agent = await setup_voice_pipeline_agent( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/agent_utils.py", line 264, in setup_voice_pipeline_agent livekit_agent | llm=get_llm_by_provider(llm_provider, converse_type), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 504, in get_llm_by_provider livekit_agent | return factory_fn(converse_type) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 415, in <lambda> livekit_agent | "google_default": lambda converse_type: create_google_llm_with_config( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 103, in create_google_llm_with_config livekit_agent | thinking_config=ThinkingConfig(thinking_level="LOW"), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 250, in init livekit_agent | validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | pydantic_core._pydantic_core.ValidationError: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden {"pid": 62, "job_id": "AJ_AtGnKEwZHXKd"} livekit_agent | 2025-12-19 200551,754 - my-worker - ERROR - [case_id:a0d9adc1-278e-4d29-96d9-8e3a38d63729] 2025-12-19 200551,754 - my-worker - ERROR - Error in entrypoint: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden livekit_agent | Traceback (most recent call last): livekit_agent | File "/llm-service/app/liveKit/agent.py", line 201, in entrypoint livekit_agent | agent = await run_voice_pipeline_agent(ctx, participant) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/agent.py", line 324, in run_voice_pipeline_agent livekit_agent | agent = await setup_voice_pipeline_agent( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/agent_utils.py", line 264, in setup_voice_pipeline_agent livekit_agent | llm=get_llm_by_provider(llm_provider, converse_type), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 504, in get_llm_by_provider livekit_agent | return factory_fn(converse_type) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 415, in <lambda> livekit_agent | "google_default": lambda converse_type: create_google_llm_with_config( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 103, in create_google_llm_with_config livekit_agent | thinking_config=ThinkingConfig(thinking_level="LOW"), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 250, in init livekit_agent | validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | pydantic_core._pydantic_core.ValidationError: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str] livekit_agent | For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden livekit_agent | 2025-12-19 200551,764 - ERROR livekit.agents - 2025-12-19 200551,764 - livekit.agents - ERROR - unhandled exception while running the job task livekit_agent | Traceback (most recent call last): livekit_agent | File "/usr/local/lib/python3.11/site-packages/opentelemetry/util/_decorator.py", line 71, in async_wrapper livekit_agent | return await func(*args, **kwargs) # type: ignore livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/usr/local/lib/python3.11/site-packages/livekit/agents/ipc/job_proc_lazy_main.py", line 240, in _traceable_entrypoint livekit_agent | await self._job_entrypoint_fnc(job_ctx) livekit_agent | File "/llm-service/app/liveKit/agent.py", line 201, in entrypoint livekit_agent | agent = await run_voice_pipeline_agent(ctx, participant) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/agent.py", line 324, in run_voice_pipeline_agent livekit_agent | agent = await setup_voice_pipeline_agent( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/agent_utils.py", line 264, in setup_voice_pipeline_agent livekit_agent | llm=get_llm_by_provider(llm_provider, converse_type), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 504, in get_llm_by_provider livekit_agent | return factory_fn(converse_type) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 415, in <lambda> livekit_agent | "google_default": lambda converse_type: create_google_llm_with_config( livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/llm-service/app/liveKit/utils/llm_factory.py", line 103, in create_google_llm_with_config livekit_agent | thinking_config=ThinkingConfig(thinking_level="LOW"), livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 250, in init livekit_agent | validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self) livekit_agent | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ livekit_agent | pydantic_core._pydantic_core.ValidationError: 1 validation error for ThinkingConfig livekit_agent | thinking_level livekit_agent | Extra inputs are not permitted [type=extra_forbidden, input_value='LOW', input_type=str]

Thanks — I see the validation error: you passed

thinking_level="LOW"

but

ThinkingConfig

doesn’t accept

thinking_level

(Pydantic rejects it). Use

include_thoughts

instead. Quick corrected patch for your factory:

from google.genai import types
from livekit.plugins import google
from google.genai.types import ThinkingConfig

def create_google_llm_with_config(converse_type: str | None = None) -> google.LLM:
    base_kwargs = dict(
        model="gemini-3-flash-preview",
        temperature=0.7,
        top_p=0.9,
        top_k=40,
    )

    if converse_type in ("singlePrompt", "singlePrompt2"):
        base_kwargs.update(
            # disable model verbalizing internal thoughts
            thinking_config=ThinkingConfig(include_thoughts=False)
        )
        return google.LLM(**base_kwargs)
    else:
        safety_settings = [
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
            types.SafetySetting(
                category=types.HarmCategory.HARM_CATEGORY_HARASSMENT,
                threshold=types.HarmBlockThreshold.OFF,
            ),
        ]

        base_kwargs.update(
            thinking_config=ThinkingConfig(include_thoughts=False),
            safety_settings=safety_settings,
        )
        return google.LLM(**base_kwargs)

If you want the model to emit its thoughts (e.g., forwarded as transcripts), set

include_thoughts=True

. Want me to show an example switching to a Gemini model that explicitly supports thinking (like the 2.5 flash native audio preview)? Sources: Gemini Live API plugin | LiveKit docs