https://livekit.io logo
Join Slack
Powered by
can we switch voice when using openai realtime mod...
# ask-ai
a
can we switch voice when using openai realtime model?
t
Thanks for reaching out, Namit. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I'm happy to help you switch the voice when using the OpenAI real-time model in LiveKit. You have two friendly options to do this: either right when you start or anytime during your session! 1. Initialization with a Specific Voice: When you create an instance of 
RealtimeModel
, just set the 
voice
parameter to the voice you’d like. For example:
Copy code
from livekit.plugins import openai

model = openai.realtime.RealtimeModel(voice="shimmer")
Some great voices you can choose from are 
alloy
, 
shimmer
, 
echo
, 
ash
, 
ballad
, 
coral
, 
sage
, and 
verse
. 2. Dynamic Voice Switching During a Session: If you want to switch things up while your session is running, you can easily update the voice using the 
update_options
method. For example:
Copy code
# Assuming 'session' is an active RealtimeSession
session.update_options(voice="echo")
This update will take effect right away for all the responses that come after. If you have any questions or want a hand with anything else, just let me know—I’m here to help! Sources: OpenAI Realtime API integration guide | LiveKit Docs | plugins/openai/src/tts.ts | examples/voice_agents/realtime_turn_detector.py
a
say method works with realtime model
t
I wanted to share a little insight about the 
say
method—it’s not directly supported by the OpenAI Realtime model in LiveKit. But don’t worry! You can still get the same effect by pairing the Realtime model with a Text-to-Speech (TTS) plugin. Here’s a friendly step-by-step on how to make it work: 1. Set up the OpenAI Realtime model with 
modalities=["text"]
so it focuses on outputting text. 2. Add a TTS plugin like 
elevenlabs.TTS()
, 
cartesia.TTS()
, or 
openai.TTS()
in your session configuration to handle turning text into speech. For example, in Python, it might look like this:
Copy code
from livekit.agents import AgentSession
from livekit.plugins import openai, elevenlabs

# Use OpenAI Realtime for text-only output
llm_plugin = openai.realtime.RealtimeModel(modalities=["text"])

# Add a TTS plugin for voice synthesis
tts_plugin = elevenlabs.TTS()

session = AgentSession(
    llm=llm_plugin,
    tts=tts_plugin
)

# Use the `say` method from the VoiceAgent context
session.say("Hello, this message will be spoken via the TTS plugin.")
This way, the Realtime model handles the text generation, and the TTS plugin brings those words to life with audio—giving you that 
say
functionality you’re looking for. If anything feels unclear or you want to dive deeper, just let me know—I’m here to help! Sources: Realtime model integrations | LiveKit Docs | OpenAI Realtime API integration guide | LiveKit Docs | OpenAI Realtime API integration guide | LiveKit Docs | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/real...
a
how to use generate_reply in its place?
t
Thanks for reaching out! Could you please clarify what you mean by using 
generate_reply
instead of the 
say
method? Are you aiming to create text responses only, or are you hoping to combine it with a Text-to-Speech (TTS) plugin to get audio output? I'm here to help you figure this out!
a
not want to use tts model
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
Open in Slack
PreviousNext
Powered by