I am using chirp3 stt model and it brings high latency aroun LiveKit Community #ask-ai

I am using chirp3 stt model, and it brings high la...

cool-doctor-14149

12/30/2025, 7:01 PM

I am using chirp3 stt model, and it brings high latency, around 1400-1800ms, is were anything we can do to make it work faster?

Copy code

session = AgentSession(
    vad=ctx.proc.userdata["vad"],
    llm=google.LLM(model="gemini-2.0-flash"),
    stt=google.STT(
        model="chirp_3",
        location="eu",
        languages=["en-US"],
        min_confidence_threshold=0.0,
        enable_word_confidence=False,
        interim_results=True,
        enable_word_time_offsets=False,
    ),
    tts=google.TTS(
        language="en-US",
        gender="female",
        location="eu",
        voice_name="en-US-Chirp3-HD-Aoede",
        use_streaming=True,
    ),
    turn_detection="vad",
    min_interruption_duration=0.5,
    min_endpointing_delay=0.3,
    max_endpointing_delay=1.0,
    preemptive_generation=True,
    resume_false_interruption=True,
)

For location, I am located in EU

tall-airline-16756

12/30/2025, 7:01 PM

Thanks for reaching out, Gediminas. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

tall-airline-16756

12/30/2025, 7:02 PM

Thanks — I’m responding to your latency question for the

chirp_3

STT setup in the EU (1400–1800 ms). A few targeted checks and tweaks can usually bring that down. Quick checklist (apply one or more): • Ensure region proximity — deploy your agent/server in the same Google Cloud region as the STT endpoint (e.g., a Europe region like

europe-west1

). Network RTT is often the largest contributor to ~1s+ latency. • Confirm model name & try alternatives —

chirp_3

may differ from the standard

chirp

model. Try

chirp

(or the model Google recommends for low-latency) and compare. • Keep streaming + process interim results — you already have `interim_results=True`; ensure your app consumes and acts on interim transcripts immediately instead of waiting for final results. • Keep endpointing low — your

min_endpointing_delay=0.3

is already low (good). Don’t increase it. • Minimize STT features (already done) —

enable_word_confidence=False

and

enable_word_time_offsets=False

reduce processing and payload size; keep them off if you don’t need them. • Co-locate LLM and TTS —

preemptive_generation=True

helps only if your Gemini model and TTS are also low-latency and hosted near the STT. If LLM response is slow, perceived overall latency won’t improve. • Benchmark to find the bottleneck — measure: (1) audio upload/network RTT, (2) STT server processing time, (3) LLM TTFT, (4) TTS time. This tells you whether STT is truly the biggest contributor. • Consider lower-latency STT providers — if after region and model tuning STT is still high, test other providers (e.g., Cartesia or a lightweight Whisper deployment) and compare real-time latency. If you want, I can help with next steps: • Run a quick test plan to isolate STT vs network vs LLM latency (I can list exact timing points to capture). • Help you try

chirp

chirp_3

5 Views

Open in Slack

Previous Next