https://livekit.io logo
Join Slack
Powered by
When using livekit realtime API (OpenAI) pipeline,...
# ask-ai
m
When using livekit realtime API (OpenAI) pipeline, how do I enforce specific setting-selected language for the LLM to communicate with the user? What are the possible ways to inject the language, which the realtime API should be strictly set to?
t
Thanks for reaching out, Rokas. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Hey there! If you want to make sure your LLM communication sticks to a specific language using the LiveKit real-time API pipeline, here’s a simple way to do it: 1. Configure the STT Plugin: Set the language in your Speech-to-Text (STT) plugin and turn off automatic language detection. For example: 
stt = deepgram.STT(model="nova-2", language="es", detect_language=False)
2. Update the TTS Plugin: Make sure your Text-to-Speech (TTS) plugin uses the same language. For example: 
tts = elevenlabs.TTS(model="eleven_turbo_v2_5", language="es")
3. Disable Language Detection: Be sure to explicitly set 
detect_language
to 
False
in the STT setup to keep things consistent. 4. Provider-Specific Configurations: Double-check that both your STT and TTS plugins support the language you picked, and that it works smoothly with your LLM model. Here’s a quick example to make it clearer:
Copy code
from livekit.agents import AgentSession
from livekit.plugins import deepgram, elevenlabs

# Set STT to Spanish
stt = deepgram.STT(model="nova-2", language="es", detect_language=False)
# Set TTS to Spanish
tts = elevenlabs.TTS(model="eleven_turbo_v2_5", language="es")

session = AgentSession(
    turn_detection=MultilingualModel(),
    stt=stt,
    tts=tts,
    # ... other configurations
)
This way, your language settings stay consistent all through the pipeline. If you have any questions or want me to walk you through it, just let me know—I’m here to help! Sources: LiveKit turn detector plugin | LiveKit Docs | pipeline-tts/elevenlabs_change_language.py | livekit-agents/livekit/agents/voice/agent_activity.py
m
I am using OpenAI's Realtime API.
t
If you want to set a specific language for LLM communication using OpenAI's Realtime API in LiveKit, you can easily do this by configuring the language in the input audio transcription settings. Here’s a simple way to get it done: 1. Set Language in Input Audio Transcription: When you initialize the RealtimeModel, just specify the language like this:
Copy code
from livekit.plugins.openai import realtime
from openai.types.beta.realtime.session import InputAudioTranscription

model = realtime.RealtimeModel(
    input_audio_transcription=InputAudioTranscription(
        model="gpt-4o-mini-transcribe",
        language="es"  # Spanish language code
    )
)
2. Update Language Dynamically: Want to switch languages during your session? No problem! Use the 
update_options
method like this:
Copy code
session = model.session()
session.update_options(
    input_audio_transcription=InputAudioTranscription(
        model="gpt-4o-mini-transcribe", 
        language="fr"  # French language code
    )
)
Just make sure the model supports the language you choose and use the standard language codes (like "en" for English, "es" for Spanish). If you hit any bumps or have more questions, feel free to ask—I’m here to help! Sources: livekit-plugins/livekit-plugins-openai/README.md
3 Views
Open in Slack
PreviousNext
Powered by