i would like to try using gemini 3.1 flash tts model as described in livekit blog: https://livekit.com/blog/gemini-3.1-flash-tts-prompting-guide?utm_source=youtube&utm_medium=video&utm_campaign=devrel&utm_content=1eez-3fmYjI but can not make it working

Thanks for reaching out, gediminas. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

this is my tts config

tts=google.TTS(
    model_name="gemini-3.1-flash-tts-preview",
    voice_name="Sulafat",
    prompt=TTS_STYLE_PROMPT,
    **(
        {"credentials_info": google_tts_credentials}
        if google_tts_credentials
        else {}
    ),
),

Thanks — and thanks for sharing the config. Quick note:

gemini-3.1-flash-tts-preview

is not listed as a supported TTS model for the LiveKit

GeminiTTS

plugin. The plugin defaults to models like

gemini-2.5-flash-preview-tts

, and LiveKit Inference lists Gemini 3.1 variants (e.g.,

gemini-3.1-flash-lite-preview

) but does not show a dedicated Gemini 3.1 Flash TTS model. That mismatch is likely why it fails. Next steps I can help with — please pick or provide: • Try a supported model first (example): set

model_name="gemini-2.5-flash-preview-tts"

or

model_name="gemini-3.1-flash-lite-preview"

and test again. • If you prefer to keep

gemini-3.1-flash-tts-preview

, share the exact error logs you see (auth, model load, or synthesis errors). • Also share runtime (Python or Node) and the LiveKit agent/plugin version. If you paste the error output and your runtime/version, I’ll give the precise change or troubleshooting steps. , with interactive elements

Question for LiveKit Support —

APITimeoutError

mid-stream with

gemini-3.1-flash-tts-preview

via

google.TTS

Summary When using

livekit.plugins.google.TTS

(Cloud Text-to-Speech,

model_name="gemini-3.1-flash-tts-preview"

) in streaming mode (

use_streaming=True

, the default) inside an

AgentSession

, the synthesis stream consistently fails with

APITimeoutError

after a non-trivial amount of audio has already been streamed to the user. This only reproduces on long LLM replies (more than a few sentences); short replies complete cleanly. The same configuration with

gemini-2.5-flash-tts

works without timeouts. We would like guidance on the recommended way to use

gemini-3.1-flash-tts-preview

with the Cloud TTS streaming path inside

AgentSession

— specifically whether the inter-chunk timeout for the streaming gRPC path is configurable on a per-

google.TTS

basis, whether disabling streaming is the recommended fallback for the 3.1 preview model, and whether there is a known server-side jitter pattern with the preview model that the plugin should be expected to absorb. Environment •

livekit-agents==1.5.6

•

livekit-plugins-google==1.5.6

•

livekit-plugins-soniox==1.5.6

•

livekit-plugins-noise-cancellation==0.2.5

• Python 3.12, macOS • `livekit-agents`Python SDK (Cloud project, not self-hosted) Configuration

from dotenv import load_dotenv
import json
import os

from livekit import agents
from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    TurnHandlingOptions,
    room_io,
)
from livekit.plugins import google, noise_cancellation, silero, soniox
from livekit.plugins.turn_detector.multilingual import MultilingualModel

load_dotenv(".env.local")


def _load_google_tts_credentials() -> dict | None:
    path = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")
    if not path:
        return None
    with open(path) as f:
        return json.load(f)


google_tts_credentials = _load_google_tts_credentials()


# Canonical Gemini 3.1 Flash TTS prompt structure (preamble + AUDIO PROFILE
# + SCENE + PERFORMANCE + CONTEXT + #### TRANSCRIPT). Trimmed for brevity.
TTS_STYLE_PROMPT = """\
Synthesize speech for the performance defined below. The profile, scene,
performance notes, and context are direction only. Do NOT speak them.
Speak ONLY the lines under #### TRANSCRIPT.

# AUDIO PROFILE: Maria J.
## "The Senior Support Rep"

## SCENE: Morning at the support desk
Bright office light, coffee cup at the corner of the desk. Maria leans
in to the mic, ready to listen.

### PERFORMANCE
Style: Warm and sincere. Voice carries feeling, not flatness.
Pace: Natural and unhurried, with a small breath before the first word.
Accent: Neutral North American.

### CONTEXT
Maria is the rep callers ask for by name.

#### TRANSCRIPT
"""


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are Maria, a support rep. Reply in 1-2 short sentences."
        )


server = AgentServer()


@server.rtc_session(agent_name="my-agent")
async def entrypoint(ctx: agents.JobContext) -> None:
    session = AgentSession(
        stt=soniox.STT(
            params=soniox.STTOptions(
                model="stt-rt-v4",
                language_hints=["en"],
                language_hints_strict=True,
                enable_language_identification=False,
            ),
        ),
        llm="openai/gpt-5.3-chat-latest",
        tts=google.TTS(
            model_name="gemini-3.1-flash-tts-preview",
            voice_name="Sulafat",
            prompt=TTS_STYLE_PROMPT,
            **(
                {"credentials_info": google_tts_credentials}
                if google_tts_credentials
                else {}
            ),
        ),
        vad=silero.VAD.load(),
        turn_handling=TurnHandlingOptions(
            turn_detection=MultilingualModel(),
        ),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_options=room_io.RoomOptions(
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=noise_cancellation.BVC(),
            ),
        ),
    )

    await session.generate_reply(
        instructions="Greet the caller and ask how you can help."
    )


if __name__ == "__main__":
    agents.cli.run_app(server)

Auth is a Google Cloud service-account JSON loaded from

GOOGLE_APPLICATION_CREDENTIALS

and passed to the plugin via

credentials_info=

. The Cloud TTS API is enabled on the project and short-reply synthesis works correctly, so credentials/permissions are not the issue. Reproduction 1. Start the agent with`uv run src/agent.py dev`and connect via Agent Console. 2. Greet the agent (it replies in 1–2 sentences — works fine). 3. Ask the agent to produce a long reply, e.g."Please tell me a long fairy tale." 4. The LLM produces a multi-paragraph response. 5. The TTS streaming session begins, audio plays correctly for ~16–17 s, then the gRPC stream stalls and`APITimeoutError`is raised. Audio playback stops mid-utterance. The same flow with

model_name="gemini-2.5-flash-tts"

(everything else identical) does not reproduce the timeout. Observed Symptom Selected log lines (one occurrence — pattern is deterministic for long replies):

ERROR  livekit.agents  TTS failed after partial audio was already sent to the user, skip retrying.
{"tts": "livekit.plugins.google.tts.TTS", "streamed": true,
 "pushed_duration": 17.089999999999975, ...}

ERROR  livekit.agents  Error in _tts_inference_task
livekit.agents._exceptions.APITimeoutError: Request timed out.

Stack trace (truncated to the relevant frames):

File ".../livekit/plugins/google/tts.py", line 457, in _run_stream
    raise APITimeoutError() from None
File ".../livekit/plugins/google/tts.py", line 405, in _run_segments
    await self._run_stream(input_stream, output_emitter, streaming_config)
File ".../livekit/plugins/google/tts.py", line 412, in _run
    await asyncio.gather(*tasks)
File ".../livekit/agents/tts/tts.py", line 479, in _main_task
    await self._run(output_emitter)
File ".../livekit/agents/voice/generation.py", line 284, in _tts_node_inference
    async for audio_frame in tts_node:

Earlier in the same session, even on the initial short greeting, this debug line appears, suggesting that some inter-chunk pauses are noticeable even on fast replies and only become fatal once enough chunks accumulate:

DEBUG  livekit.agents  flush audio emitter due to slow audio generation

Diagnosis (best understanding so far) Reading

livekit-plugins-google==1.5.6

source: • `google.TTS.__init__`builds a`SynthesizeStream`when`use_streaming=True`. • `SynthesizeStream._run_stream`opens a bidirectional gRPC stream against`texttospeech.googleapis.com`via`streaming_synthesize`and consumes chunks with`async for resp in stream`. • The deadline is governed by`self._conn_options.timeout`, which comes from`DEFAULT_API_CONNECT_OPTIONS`. The failure pattern is therefore: the gRPC stream remains open and has been delivering audio chunks, but at some point the next chunk does not arrive within

conn_options.timeout

seconds. This is consistent with server-side inter-chunk jitter on

gemini-3.1-flash-tts-preview

— long replies accumulate enough chunks that hitting one stall is essentially guaranteed, while short replies finish before any stall happens. We do not see any `4xx`/`5xx`; the gRPC stream is simply silent for long enough to trip the deadline. Concrete Questions 1. Is

gemini-3.1-flash-tts-preview

officially supported by

livekit-plugins-google.TTS

in the streaming path today?It is listed in`livekit/plugins/google/models.py::GeminiTTSModels`, but the Gemini TTS plugin guide page (

/agents/models/tts/gemini/

) only shows`google.beta.GeminiTTS`examples with`gemini-2.5-flash-preview-tts`. Is the recommended path for 3.1 preview`google.TTS`(Cloud TTS) or`google.beta.GeminiTTS`(Gemini Developer API)? 2. Is there a way to override the per-chunk recv deadline for the streaming gRPC call from the agent codewithout reaching into the plugin internals?`google.TTS.__init__`does not expose`conn_options`. Is the supported pattern`tts.with_options(conn_options=APIConnectOptions(timeout=N))`(we did not see this method on the base`TTS`class), or something else? An example of bumping the timeout to e.g. 30 s would be very helpful. 3. Is

use_streaming=False

the recommended fallback for

gemini-3.1-flash-tts-preview

until the model exits preview?If we set`use_streaming=False`, the plugin uses`synthesize_speech`(chunked), which we expect will not exhibit inter-chunk jitter at the cost of higher time-to-first-audio. Is there guidance on this trade-off? 4.

google.beta.GeminiTTS

(Gemini Developer API) currently uses non-streaming

client.aio.models.generate_content(...)

internally, which produces high time-to-first-byte for long replies. The Google AI Studio sample for the same model uses`client.models.generate_content_stream(...)`(yields audio chunks as they're generated). Is there a plan to add a streaming variant of`google.beta.GeminiTTS`that uses`generate_content_stream`so we get low TTFB without the Cloud TTS gRPC stall pattern? 5. Is there a known issue / open ticketfor the Cloud TTS streaming behaviour we are seeing with the 3.1 preview model that we can subscribe to or follow? What we have already tried VariantResult`google.TTS(use_streaming=True)` (default), 3.1 preview, long reply`APITimeoutError` at

pushed_duration ~17 s

(this report)`google.TTS`, 2.5 GA model, long replyWorks, no timeout`google.beta.GeminiTTS`, 3.1 previewNo timeout, but very high TTFB on long replies (uses non-streaming

generate_content

)Custom subclass of

google.beta.GeminiTTS

using `generate_content_stream`Lower TTFB but unsupported / ad-hoc; want to know if there is a first-class path What we want The smallest, supported configuration of

livekit.plugins.google

that: • uses`gemini-3.1-flash-tts-preview`(required by the project), • delivers low time-to-first-audio (≤ 1 s after LLM produces the first sentence), • does not raise`APITimeoutError`on multi-paragraph LLM replies in`AgentSession`. Any guidance — even just "set

use_streaming=False

and override

conn_options.timeout

like X" — would unblock us. Thank you!

Thanks — this is very helpful detail, and you're right to suspect an inter-chunk deadline. I don’t have a definitive doc-backed fix for the exact APITimeoutError you’re seeing, but I can help narrow this down and recommend safe workarounds. What I’m responding to: the TTS streaming timeout mid-utterance for

gemini-3.1-flash-tts-preview

using

google.TTS

with

use_streaming=True

. Immediate recommendations (fastest to try) • Try disabling streaming to confirm the deadline is the issue:

tts = google.TTS(..., model_name="gemini-3.1-flash-tts-preview", use_streaming=False, ...)

If long replies complete with

use_streaming=False

, the problem is the streaming gRPC deadline/idle window. • If you need low TTFA and can’t accept non-streaming, test the Gemini Developer API path: ◦ Use

google.beta.GeminiTTS(model="gemini-3.1-flash-tts-preview", ...)

. It currently uses non-streaming

generate_content()

(higher TTFA) but avoids the Cloud TTS gRPC streaming deadline. Required info to diagnose and give a concrete fix Please provide the following so I can recommend the exact change or escalate with full evidence: 1. Full DEBUG logs around the failure (enable DEBUG for livekit and google plugin) including any gRPC/deadline lines. 2. The exact pip package outputs (confirm

livekit-agents==1.5.6

and

livekit-plugins-google==1.5.6

) and Python patch level. 3. Result of toggling

use_streaming=False

(did long replies finish or also fail?). 4. Minimal reproducible script that calls

google.TTS

streaming directly (outside AgentSession), plus its logs — this isolates plugin vs AgentSession orchestration. 5. Any additional error fields from the APITimeoutError (if present) or gRPC metadata. Short answers to your concrete questions (based on current package behavior) • Is

gemini-3.1-flash-tts-preview

officially supported in the Cloud TTS streaming path? It appears in internal model lists but Cloud TTS streaming is not the intended Gemini path; the recommended path for Gemini models is the Gemini plugin (

google.beta.GeminiTTS

). • Can you override the per-chunk recv deadline from agent code? The

google.TTS

initializer does not expose

conn_options

or a public timeout override in the current plugin API. • Is

use_streaming=False

the recommended fallback? Yes — it avoids inter-chunk gRPC idle deadlines at the cost of higher time-to-first-audio. • Is there a streaming variant planned for

google.beta.GeminiTTS

that uses

generate_content_stream

? There is no documented plan in the public docs I can reference. • Is there a known ticket for Cloud TTS streaming jitter with 3.1 preview? No public ticket or doc is available. Next step from me after you share logs/snippets • If logs show the stream simply idle until the deadline, I’ll recommend a concrete short-term pattern (use_streaming=False for reliability, or switch to

google.beta.GeminiTTS

and accept higher TTFA). • If you confirm you need low TTFA and want me to escalate with a reproducible case and logs, say “please escalate” and I’ll prepare a concise reproduction bundle and request engineering review. Which of the following do you want to try first? • I’ll wait for the DEBUG logs and reproduction script to analyze further. • Or test

use_streaming=False

now and report results. Sources: Gemini TTS plugin guide | LiveKit Documentation | Google Cloud TTS integration guide | LiveKit Documentation | Gemini Live API plugin | LiveKit Documentation | Amazon Nova Sonic integration guide | LiveKit Documentation | Text-to-speech (TTS) models overview | LiveKit Documentation, with interactive elements