https://livekit.io logo
Join Slack
Powered by
I want to make my language switching robust, once ...
# ask-ai
c
I want to make my language switching robust, once the tool is called, does the tts alway use the new language or will it switch back to default?
t
Thanks for reaching out, Subham. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I totally understand how important it is to have the TTS language set just the way you want it. When you switch the TTS language in LiveKit, it actually stays set to that language and won’t go back to the default unless you decide to change it again. This way, every time you use TTS afterward, it will consistently use the new language setting you picked. If you have any other questions or need help with this, I’m here for you! Sources: pipeline-tts/elevenlabs_change_language.py | livekit-plugins/livekit-plugins-google/livekit/plugins/google/tts.py | Spitch TTS integration guide | LiveKit Docs | Sending text | LiveKit Docs
c
But my agent goes back to talking in english, can I update the stt language too>?
t
Absolutely! You can update the STT language dynamically in LiveKit using the 
update_options
method. This means you can switch the transcription language on the fly without having to restart your session. If you'd like, I’m here to help you step-by-step with how to set this up for your specific needs—just let me know! Sources: Gladia integration guide | LiveKit Docs | Gladia integration guide | LiveKit Docs | livekit-plugins/livekit-plugins-google/livekit/plugins/google/stt.py
c
can you give an example for the same?
t
I'd love to share a helpful example of how you can dynamically update the STT language using the 
update_options
method:
Copy code
# Example for Gladia STT plugin
from gladia import STT

gladia_stt = STT()
# Update the transcription language to Japanese and English
gladia_stt.update_options(languages=["ja", "en"])
This way, you’re setting the transcription language to Japanese and English. If you'd like, I’m here to help with an example for a specific STT plugin or any other questions you might have—just let me know!
c
2025-09-22 114705,122 - ERROR livekit.agents - exception occurred while executing tool Traceback (most recent call last): File "/Users/subhamdas/revvspot_staging/agent_builder/.venv/lib/python3.12/site-packages/livekit/agents/voice/generation.py", line 516, in _traceable_fnc_tool val = await function_callable() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/subhamdas/revvspot_staging/agent_builder/src/agent.py", line 305, in language_detection self.stt.update_options(language_code=language_code) ^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'STT' object has no attribute 'update_options' {"room": "playground-tXLe-61Iu", "function": "language_detection", "speech_id": "speech_043286be75e3", "pid": 66584, "job_id": "AJ_GJWCbVtCwCz4"} this is what I got when i tried to incoporate this as a tool call
t
It looks like the error is happening because the 
update_options
method isn’t available on the base 
STT
class. To fix this, you’ll want to use a specific STT plugin like Gladia, Deepgram, or Google, which includes this method. Here’s a friendly example using Gladia:
Copy code
from livekit.plugins import gladia

# Instantiate the Gladia STT plugin
stt = gladia.STT()

# Update the transcription language dynamically
stt.update_options(languages=["ja", "en"])
Make sure you’re importing and using the right plugin for your setup. If you’d like, just let me know which STT provider you’re working with, and I’d be happy to help you further! Sources: STT | LiveKit Agents | livekit-plugins/livekit-plugins-google/livekit/plugins/google/tts.py | Gladia integration guide | LiveKit Docs
c
but i want to do it inside a tool call, how to implement this? How will my tool call inside the agent use update_options?
t
I'd love to help you implement 
update_options
within a tool call inside your agent! To get a clearer picture and offer the best guidance, could you share a bit more about how your agent is set up? For instance: • Are you using a specific STT plugin like Gladia or Deepgram? • How are you currently executing the tool call in your agent? • Do you happen to have an example of your current implementation? With these details, I’ll be able to provide a solution that fits your needs perfectly. Looking forward to your reply!
c
yes,
Copy code
class LivspaceInboundAgent(Agent):
    def __init__(self,
                 chat_ctx=None,
                 user_project_details=None,
                 dial_info=dict[str, Any]):
        self.__name__ = "livekit_livspace_inbound"
        instructions = INSTRUCTIONS
        super().__init__(
            instructions=instructions,
            # stt=elevenlabs.STT(),
            stt=deepgram.STT(),
            llm=google.LLM(model="gemini-2.5-flash-lite"),
            tts=elevenlabs.TTS(
                model="eleven_flash_v2_5", 
                voice_id="H8bdWZHK2OgZwTN7ponr",
                voice_settings=elevenlabs.VoiceSettings(
                    stability=0.5,
                    similarity_boost=0.7,
                    speed=1.10,
                ),
                streaming_latency=4
                ),
            turn_detection=MultilingualModel(),
            chat_ctx=chat_ctx,
        )
        self.dial_info = dial_info
        self.user_project_details = user_project_details
        self.participant: rtc.RemoteParticipant | None = None

@function_tool
    async def language_detection(self, context: RunContext, language_code: str):
        """
        Change the conversation language when the user expresses a language preference explicitly (Hindi/English)
        Call this function when:
        - Direct requests: "Can we speak in Hindi?", "Switch to Hindi", "Let's continue in Hindi"
        - Questions about capability: "Do you speak Hindi?", "क्या आप हिंदी में बात करते हैं?"
        - Stated preferences: "I would prefer Hindi", "Hindi would be better for me"

        """
        <http://logger.info|logger.info>(f"Language detection function called with language: {language_code}")

        # Map language codes to proper format for each service
        language_mapping = {
            "en": {"deepgram": "en-US", "elevenlabs": "en"},
            "hi": {"deepgram": "hi", "elevenlabs": "hi"}
        }
        
        if language_code not in language_mapping:
            logger.warning(f"Unsupported language code: {language_code}")
            return f"Sorry, I don't support the language code '{language_code}'. I can only speak English (en) or Hindi (hi)."
        
        # Update STT (Speech-to-Text) language
        if self.stt is not None:
            deepgram_lang = language_mapping[language_code]["deepgram"]
            self.stt.update_options(language=deepgram_lang)
            <http://logger.info|logger.info>(f"Updated STT language to: {deepgram_lang}")

        # Update TTS (Text-to-Speech) language
        if self.tts is not None:
            elevenlabs_lang = language_mapping[language_code]["elevenlabs"]
            self.tts.update_options(language=elevenlabs_lang)
            <http://logger.info|logger.info>(f"Updated TTS language to: {elevenlabs_lang}")
            
        return f"Language switched to {'Hindi' if language_code == 'hi' else 'English'}. I will now respond in this language."
t
Thank you so much for sharing the implementation details! From what I see in your code, it looks like the hiccup might be with the 
update_options
method call for your STT instance. Since you're using 
deepgram.STT()
, it's a good idea to double-check that the 
update_options
method is properly implemented and supports the 
language
parameter. Here’s a friendly checklist to help you verify and tweak your implementation: 1. Make sure the 
deepgram.STT
plugin is correctly imported and initialized. 2. Check that the 
update_options
method in 
deepgram.STT
actually supports the 
language
parameter. If it doesn’t, the plugin documentation might have the right method or parameter name you need. 3. Try testing the 
update_options
method on its own to see if it updates the language as expected:
Copy code
from livekit.plugins import deepgram

stt = deepgram.STT()
stt.update_options(language="en-US")
If this standalone test works but you still face issues when calling it within your tool, there could be something up with how the 
self.stt
instance is being handled or passed around in your agent. Please feel free to reach out if you want me to walk you through it or if you need any more help—I’m here for you!
c
does elevenlabs stt plugin have update_options?
t
Thank you for reaching out! I wanted to let you know that the ElevenLabs STT plugin doesn’t support the 
update_options
method. This means that configuration options, like the language, need to be set when you first initialize the plugin and can’t be changed on the fly. If you’re looking for dynamic language switching, I’d recommend trying a plugin like Deepgram, which does support 
update_options
. I hope this helps, and please feel free to ask if you have any more questions! Sources: livekit-plugins/livekit-plugins-azure/livekit/plugins/azure/stt.py | STT | LiveKit Agents
c
If I change the language for stt, does it mean that it will try to decode the message in that language only?
t
Thanks for reaching out! When you change the STT language in LiveKit, it usually sets the transcription engine to focus on that specific language. That said, the exact behavior can vary depending on the STT plugin you're using. Some plugins might decode only in the language you set, while others could still support detecting multiple languages if they have that feature. If you let me know which STT plugin you're working with, I’d be happy to give you more detailed guidance!
c
What happens if I have set language to hindi but the user talks in english, how will the information go to the user agent?
t
I understand that if the STT language is set to Hindi but the user speaks in English, the transcription accuracy might not be perfect. This happens because most STT engines focus on the configured language, which can lead to some challenges if the spoken language is different. To help with this, you might consider: • Enabling multi-language detection if your STT plugin supports it. • Setting up a fallback mechanism that can detect and switch languages dynamically based on what the user says. Please feel free to reach out if you'd like some guidance on how to implement these solutions or if you want to share more about the STT plugin you're using. I'm here to help!
c
why does my agent say lines in english even after I have changed the language to hindi?
t
I understand it can be a bit frustrating if your agent keeps speaking in English even after changing the TTS language to Hindi. Let’s explore a few possibilities together: • Sometimes, the TTS language change might not have been applied successfully. Could you please double-check if the 
update_options
method for the TTS plugin was called correctly and that the language parameter was indeed updated to Hindi? • It’s also possible that the text being sent to the TTS engine is still in English. It would be great to verify if the input text is being translated or generated in Hindi before it reaches the TTS engine. • Another thing to consider is whether the TTS plugin fully supports Hindi or if it needs some extra configuration to produce the correct language output. Please feel free to share more details about your setup or let me know if you’d like me to help you troubleshoot any of these steps. I’m here to assist you!
c
How to ensure the text produced stays in Hindi before reaching the TTS machine?
t
To make sure the text stays in Hindi before it reaches the TTS engine, here are a few friendly tips you can try: 1. Set a system prompt for the LLM: You can include simple instructions like "Always respond in Hindi" to gently guide the LLM to generate text in Hindi. 2. Use the 
before_tts_cb
callback: This is a handy way to check or tweak the text right before it goes to the TTS engine. For instance, you might add a translation service here to ensure the text is perfectly in Hindi. 3. Configure the TTS plugin: Don’t forget to set the 
target_language_code
parameter to Hindi (like 
hi-IN
) so the TTS engine speaks in Hindi as expected. If you'd like, I’m here to help you step-by-step with these or assist in setting up a translation service. Just let me know! Sources: Sarvam TTS integration guide | LiveKit Docs | Sarvam TTS integration guide | LiveKit Docs | Agents 0.x migration guide | LiveKit Docs | VoicePipelineAgent | LiveKit Docs | Pipeline nodes and hooks | LiveKit Docs
c
Use the 
before_tts_cb
callback explain this
t
The 
before_tts_cb
callback in LiveKit is a wonderful tool that lets you tweak or check the text generated by the LLM before it reaches the TTS engine. This can be really helpful to make sure the text is in the right language, tidy up the formatting, or make any other adjustments you might need. Here’s a friendly example to show you how it works:
Copy code
def before_tts_callback(agent, text: str):
    # Example: Ensure the text is in Hindi (you can integrate a translation service here)
    if not is_hindi(text):
        text = translate_to_hindi(text)  # Custom function to translate text to Hindi
    return text

agent = VoicePipelineAgent(
    # Other configurations...
    before_tts_cb=before_tts_callback,
)
In this example, the callback kindly checks if the text is in Hindi and translates it if needed. If you’d like, I’m here to help you set this up or assist with integrating a translation service—just let me know! Sources: chain-of-thought-tts/agent.py | VoicePipelineAgent | LiveKit Docs | VoicePipelineAgent | LiveKit Docs
c
Is there a way we can dynamically decide to use this function or not, i mean the agent can decide it?
t
Absolutely! You can definitely decide on the fly whether to use the 
before_tts_cb
function by adding some conditional logic right inside the callback. For instance, you might want to check certain things like the language or the context, and then tweak the text only when it makes sense:
Copy code
def before_tts_callback(agent, text: str):
    # Example: Only modify text if the language is not Hindi
    if agent.current_language != "hi":
        text = translate_to_hindi(text)  # Custom function to translate text to Hindi
    return text

agent = VoicePipelineAgent(
    # Other configurations...
    before_tts_cb=before_tts_callback,
)
Here, the callback looks at the agent's current language and only applies the translation if it’s not Hindi. You can easily adjust this logic to fit exactly what you need. If you have any questions or want to explore this further, just let me know—I’m here to help!
c
How to use google stt?
t
To get started with using Google STT in LiveKit, here’s a simple guide to help you along the way: 1. Install the plugin: Just run 
pip install "livekit-agents[google]~=1.2"
to add the Google STT plugin to your setup. 2. Set up authentication: You’ll need to provide your Google Cloud credentials. Here are a few ways you can do this:Also, make sure the Cloud Speech-to-Text API is enabled in your Google Cloud project. ◦ Pass the credentials through the 
credentials_info
dictionary. ◦ Set the path to your JSON key file using the 
credentials_file
parameter or the 
GOOGLE_APPLICATION_CREDENTIALS
environment variable. ◦ Or simply use Application Default Credentials (ADC). 1. Initialize the STT instance:
Copy code
from livekit.plugins import google

stt_instance = google.STT(
    model="chirp",                # Use the advanced 'chirp' model
    spoken_punctuation=False,     # Disable spoken punctuation
    languages="en-US",            # Specify language
    interim_results=True          # Enable interim (partial) results
)
Integrate with your agent: You can now use the STT instance within an 
AgentSession
like this:
Copy code
session = AgentSession(
    stt=stt_instance,
    # ... other components like LLM, TTS, etc.
)
If you have any questions or need a hand with the setup, please don’t hesitate to reach out—I’m here to help! Sources: Google Cloud STT integration guide | LiveKit Docs | livekit-plugins/livekit-plugins-assemblyai/livekit/plugins/assemblyai/stt.p... | Google integration guide | LiveKit Docs | Google integration guide | LiveKit Docs | Google Cloud STT integration guide | LiveKit Docs
5 Views
Open in Slack
PreviousNext
Powered by