modern-leather-76107
11/21/2025, 5:49 AMlivekit-plugins-elevenlabs 1.3.3) is always reported as "en". That means I don’t have a reliable signal for what language the user actually spoke during the last utterance. This becomes a problem because I want to do things like:
• Track which language a session is in
• Choose the correct TTS voice for that language
• Switch dialects/accents per language where it improves the experience
I know STTv2 / Scribe v2 Realtime recently landed in a PR, and I’m hoping it will expose accurate language detection in the WebSocket stream, but I’m not sure what to expect or whether it’s available yet on PyPI.
The bigger challenge is on the TTS side. ElevenLabs STT (especially with Scribe v2) can auto-detect a very wide range of languages, but TTS voices don’t cover nearly as many. Even the best Turbo/Flash v2.5 voices support around 30–32 languages, and a single voice often doesn’t sound natural across all of them. So even if I correctly detect the spoken language, I still need a way to route the LLM’s output to the appropriate voice ID that best matches the target language, accent, and dialect. Without reliable language metadata from STT, I can’t do this cleanly.
Right now I’m considering adding lightweight language detection on the text itself (e.g., before passing to TTS), but I’m trying to avoid unnecessary latency and complexity if the STTv2 integration is meant to solve this.
My main questions for the community are:
1. With STTv2 / Scribe v2 Realtime, will LiveKit Agents expose accurate per-utterance language codes so I can reliably determine the last user language?
2. What’s the recommended way to route TTS output when one voice doesn’t cover all languages well and I need to switch between multiple voice IDs to get better coverage and accents?
3. Is there an existing pattern for mapping detected STT language → best TTS voice profile inside the LiveKit agent loop?refined-appointment-81829
11/21/2025, 2:29 PMrefined-appointment-81829
11/21/2025, 2:30 PMupdate_options()