This message was deleted.

Thanks for reaching out, Aleh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i can't use tts, only sts realtime model

Thanks — that’s a helpful constraint to know. You said you cannot use TTS and must use STS (pre-recorded speech). I can help, but I need a few details so my recommendations match your STS pipeline. A few quick clarifying questions: 1. By “STS” do you mean playing pre-recorded audio files (speech-to-speech playback), or do you mean a speech transformation pipeline (input speech → model → transformed output speech)? 2. What format is the STS audio in (e.g., WAV/MP3/Opus), sample rate, mono/stereo, and typical file size/duration? 3. How do you currently inject that audio into the session? (example:

session.say(audio=...)

, sending RTP, providing an external media URL, or another method) 4. Does your realtime session API support passing pre-recorded audio and an

allow_interruptions

flag when scheduling playback? If unsure, tell me which client/server SDK or API call you use to play audio and I’ll adapt guidance to it. 5. Do you need the voicemail audio to be dynamically generated per call (per-caller personalization) or can it be fully static/cached per message? What I can recommend immediately (STS-focused, actionable) • Use the session-level non-interruption mechanisms and disable input while playing STS: ◦ Before playback: disable silence/speech detection and set session to not create or interrupt LLM responses. ◦ If your API supports it, schedule the pre-recorded audio with an

allow_interruptions=False

flag (or equivalent) at the time you start playback — setting flags after starting playback is ineffective. ◦ If the API cannot set that flag on playback, disable any session-level auto-response generation and/or disable input audio so nothing in the pipeline can trigger a reply during playback. • Preload / cache the STS audio to avoid latency ◦ Keep the audio ready in memory or local storage to avoid network/TTS delays that could open a window for interruptions. • Increase and make configurable the post-playout drain period ◦ After the player reports playout complete, wait an additional configurable window (8–10s recommended) before hanging up to let PSTN jitter buffers flush. • Protect against unexpected re-entry ◦ After STS playback, avoid calling any LLM/response-generation methods. If you must inject a short goodbye and it must be STS, play another pre-recorded goodbye audio file rather than triggering an LLM reply. • If you can control RTP/session input: ◦ Temporarily mute or disable inbound audio from the PSTN bridge during playback so silence detection or VAD can’t create spurious events. How I’ll help next • If you tell me which of the above applies (answers to Q1–Q4), I’ll provide a concrete code patch/snippet that: ◦ Replaces your

generate_reply()

flow with the correct STS playback approach for your API, ◦ Shows where to disable inputs/turn detection and set the drain time, ◦ And demonstrates a safe hangup path that avoids re-entry. Which of the clarifying questions above can you answer now? If you prefer, paste the exact call you use to play STS audio and I’ll write the patch for that API. Sources: Uninterruptable Agent | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | Turns overview | LiveKit Documentation | Voice Agent Architecture: STT, LLM, and TTS Pipelines Explained | LiveKit | Realtime models overview | LiveKit Documentation

i can't use tts, only sts realtime model (edited)

Thanks — that clarification helps. You can get a reliable, non-interrupted voicemail even when you must use an STS (speech-to-speech / pre-recorded audio) realtime model, but I need a few specifics about how your STS audio is played so I can give a concrete patch. Quick clarifying questions (the answers let me produce a ready-to-drop code change) 1. By STS do you mean fully pre-recorded audio files you play back (static or per-caller), or a live speech transformation that synthesizes audio on the fly from an incoming model? 2. How do you currently inject STS audio into the call/session? ◦ e.g.,

session.say(audio=...)

, sending RTP into the bridge, pointing the session to an external media URL, or another custom pipeline? 1. Can your playback call accept an

allow_interruptions

(or equivalent) flag at schedule time? If you don’t know, paste the exact API/SDK call you use to start playback. 2. Audio format and properties: what format do you use for STS audio (WAV/MP3/Opus), sample rate, mono/stereo, and typical duration? 3. Is the voicemail personalized per-call (dynamic) or reused across calls (cacheable)? 4. Do you have control over input audio during playback (can you mute/disable inbound audio or disable VAD/silence detection)? Immediate STS-focused guidance you can apply right away • Schedule playback with interruption protection at the time you start it. If your API supports a boolean like

allow_interruptions=False

, use that when scheduling the STS audio. Setting protection after playback starts is ineffective. • If you cannot set an interruptions flag per-playback, disable session-level auto-responses and silence/VAD detection before playback so nothing in the session can trigger an LLM reply while the STS audio is playing. • If possible, temporarily mute or disable inbound audio on the session during playback (this prevents VAD/speech detection from firing). • Keep the STS audio preloaded/cached locally (avoid synthesizing or fetching it right before playback) to reduce start latency and the window for interruptions. • After the playback reports playout complete, wait an additional configurable drain window (8–10s recommended) before hanging up to allow PSTN jitter buffers to fully flush. • Avoid calling any LLM or

generate_reply()

after voicemail; if you need a closing line, play a second pre-recorded STS farewell audio file. If you prefer, I can: • Provide a direct code patch that replaces your current

generate_reply()

path with an STS playback flow for your specific playback API. • Recommend specific drain and timeout values (conservative vs aggressive). • Create a small carrier test checklist. Which do you want next? If you want the code patch, paste the exact API call you currently use to play STS audio (or confirm whether you use

session.say(audio=...)

, RTP, or external URL).

handle = await self.session.say(
    voicemail_message,
    allow_interruptions=False,
)
await handle.wait_for_playout()
<http://logger.info|logger.info>("Voicemail fully played back (interrupted=%s)", handle.interrupted)

# Drain PSTN buffer (configurable)
drain_sec = agent_settings.VOICEMAIL_FINAL_DRAIN_SEC  # e.g., 8
await asyncio.sleep(drain_sec)

# Restore input only if you will continue; otherwise hang up
await self.hangup(handle)

{"message": "the RealtimeModel uses a server-side turn detection, allow_interruptions cannot be False when using VoiceAgent.say(), disable turn_detection in the RealtimeModel and use VAD on the AgentTask/VoiceAgent instead", "level": "WARNING", "name": "livekit.agents", "pid": 44, "job_id": "AJ_cCXMa7tu7QBz", "room_id": "RM_uStZYCDAoC5e", "timestamp": "2026-03-16T141847.173227+00:00"}

Thanks — that warning explains the root cause: your realtime STS model performs server-side turn detection, so

session.say(..., allow_interruptions=False)

can’t give you strict client-side non-interruptible playback. I’ll keep this focused and practical so you can pick an approach that fits your pipeline. What’s happening (short) • Realtime STS models do server-side turn detection/VAD. That server-side behavior takes precedence and prevents the client from enforcing non-interruptible playback via

say()

or a post-hoc

handle.allow_interruptions = False

. • LiveKit warns when you attempt

say(..., allow_interruptions=False)

with such a model because the model controls turn-taking. Recommended options (choose one based on what you can control) 1) Use generate_reply() with carefully-written instructions (simplest) • Why: This uses the STS model as intended — the model will generate and play the audio from the server side. • How: a. Use

generate_reply(instructions=...)

to tell the model to read the voicemail clearly and fully. b. Call

await handle.wait_for_playout()

and then sleep for a drain window before hangup. • Trade-offs: You can’t fully guarantee no barge-in, but a clear, directive prompt (e.g., "read this voicemail completely without stopping") plus a drain window works well in practice. Concise example:

handle = await self.session.generate_reply(
    instructions=f"Read the following voicemail slowly and completely: '{voicemail_message}'"
)
await handle.wait_for_playout()
await asyncio.sleep(agent_settings.VOICEMAIL_FINAL_DRAIN_SEC)
await self.hangup(handle)

2) Disable server-side turn detection and use client-side VAD (most control) • Why: Turning off server-side turn detection hands turn control back to your agent so

allow_interruptions

and local VAD behave as expected. • How: a. Configure the realtime model input to disable automatic_activity_detection / server turn detection. b. Attach a client-side VAD (e.g., Silero) or LiveKit’s turn detector for controlled barge-in logic. c. Then you can play audio (or instruct model) with client-respected interruption flags. • Trade-offs: You may lose some realtime model features (and must handle VAD reliability yourself). Example config sketch:

# Pseudocode — adapt to your SDK
session = AgentSession(
    llm=RealtimeModel(realtime_input_config=RealtimeInputConfig(automatic_activity_detection=Disabled)),
    vad=silero.VAD.load(),
)
# Now schedule playback and client VAD will manage interruptions

3) Mute/disable inbound audio during playback (most robust) • Why: If you can mute or disable the session input during voicemail, nothing can trigger server-side or client-side turn detection. • How: a. Before playback:

session.input.set_audio_enabled(False)

(or equivalent). b. Start playback via

generate_reply()

(STS) or play pre-recorded audio into the session. c. Wait for playout, drain, then hang up (or re-enable input if continuing). • Trade-offs: If you plan to continue the session after voicemail and need inbound audio, you must re-enable input. Also you must ensure the muting call actually prevents server-side triggers. 4) Pre-record and publish audio track into the Room (bypass STS for playback) • Why: If the platform allows you to publish a pre-recorded track (audio RTP/track) into the Room, the STS model’s server-side turn-detection may not control that playback, giving you more reliable delivery. • How: Publish the pre-recorded voicemail as a LocalTrack or inject RTP into the bridge, then wait for playout and drain. • Trade-offs: More infrastructure effort, but highest reliability across carriers. Additional practical tips (applies to all options) • Make drain configurable and conservative: 8–10s is safer than 5s for PSTN. • Avoid calling any LLM/response-generation after voicemail unless you expect replies — that’s what causes re-entry. If you need a farewell, play another STS clip or include it in the same generate_reply instruction. • Preload or cache audio assets to remove fetch/synthesis latency windows. • Instrument: log handle.interrupted, playout timestamps, and carrier type to tune drain and turn-detection behavior. Quick diagnostic questions so I can provide a drop-in patch 1. Which STS model/provider are you using (OpenAI Realtime, Nova Sonic, etc.)? 2. Can you disable server-side turn detection in your realtime model config? If unsure, paste the code/config you use to create the realtime model/session. 3. Can you mute/disable inbound audio on the session (e.g.,

session.input.set_audio_enabled(False)

)? 4. Do you have the option to publish a pre-recorded audio track into the Room? Tell me which option you prefer or answer any of the diagnostic questions and I’ll provide a concrete code patch for your exact flow. Sources: Gemini Live API plugin | LiveKit Documentation | Turns overview | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | Realtime models overview | LiveKit Documentation | examples/voice_agents | livekit/livekit_composite | DeepWiki