i'm using agents on python with gpt realtime. It seems that when i invoke the speech using "agent_session.say()" it uses a different voice than when talking to it normally.

Thanks for reaching out, Antonio. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i actually did set openai realtime and openai tts both with marin voice, still it sound different. I don't know if it's just an impression but it seems slower when i use the say() method

Thanks — I understand you set both to "marin" but

say()

still sounds different/slower. Let's narrow this down. Possible causes • OpenAI Realtime’s built-in voice synth and the OpenAI TTS engine can implement the same voice name differently (prosody, pacing). •

agent_session.say()

TTS may have default rate/pitch settings that differ from the realtime synth. • Encoding, sample rate, or client-side playback buffering could introduce perceived slowness. Quick troubleshooting steps 1. Confirm exact config/code: please paste the

AgentSession

creation and the

say()

call (hide any secrets). 2. Check TTS params: if you pass any

rate

,

pitch

, or

speed

settings to the TTS plugin, try adjusting them (increase

rate

slightly). 3. Compare raw audio files: ◦ Save the realtime reply audio and the

say()

audio to files and compare sample rate and duration. If the

say()

file is longer for the same text, it’s a TTS pacing difference. 1. Try the same voice directly with the OpenAI TTS endpoint (outside AgentSession) to see if pacing matches

say()

— that isolates whether AgentSession adds processing. 2. Test another TTS provider (short test with e.g., Cartesia or ElevenLabs) to see if pacing matches realtime; this helps confirm whether OpenAI TTS is the source. If you share: • the

AgentSession

snippet, • the

say()

call, • and (if possible) short audio clips or durations for the same sentence from realtime vs

say()

, I can point to the exact parameter to change or recommend the next step. , with interactive elements

agent_session = AgentSession( instructions=instructions, tools=tools, vad=_vad_instance, stt=openai.STT(api_key=api_key, language=output_language), llm=openai.realtime.RealtimeModel(model=model, voice="marin", api_key=api_key, temperature=temperature), tts=openai.TTS(voice=voice, api_key=api_key), )

Thanks — I hear you: you set both Realtime and TTS to "marin" but

agent_session.say()

still sounds different and slower. That’s expected unless you route all speech through the Realtime model. Why this happens (short) • The Realtime model’s

voice="marin"

is an expressive, conversation-oriented synth. •

agent_session.say()

uses the separate TTS plugin (

openai.TTS

), which reads text more like a TTS reader and can have different pacing and prosody. • The TTS implementation of “marin” is not identical to the Realtime implementation; TTS may default to slower rate/neutral prosody. Practical options to fix • Use Realtime-only (recommended if you want exact parity): remove

tts=

so all speech is produced by

RealtimeModel(voice="marin")

. • Keep TTS but improve its pacing: ◦ Add

instructions

to your

openai.TTS

(e.g., “Speak in a warm, conversational tone with natural pacing and slight pauses”). ◦ If your TTS supports rate/pitch params, increase

rate

slightly (check your TTS client for these args). • Compare outputs programmatically: save both audio files and compare duration/sample rate to confirm it's a pacing difference. Next step for me to help • Do you want to switch to Realtime-only, or keep TTS and tune its settings? • If you want tuning, paste the

openai.TTS(...)

call you used (no secrets) and I’ll suggest the exact

instructions

or params to try. Sources: OpenAI TTS plugin guide | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | Realtime models overview | LiveKit Documentation | Sequential Pipeline Architecture for Voice Agents | LiveKit | Sequential Pipeline Architecture for Voice Agents | LiveKit, with interactive elements