steep-balloon-41261
01/02/2026, 2:30 PMtall-airline-16756
01/02/2026, 2:30 PMtall-airline-16756
01/02/2026, 2:31 PMpreemptive_generation=True, interim transcriptions may trigger regeneration and cancel playback. Disable preemptive generation to test.
• TTS streaming / network latency: Streaming TTS providers (WebSocket-based) can have buffering gaps on high latency or unstable networks. Test on a stable network or switch to a non-streaming provider to compare.
• Audio buffer/queue underrun or overflow: Internal audio frame queue issues can drop frames. Review agent audio logs for buffer warnings and test with smaller/larger buffer settings if available.
• Packet loss / WebRTC issues: Although WebRTC mitigates many problems, client-side packet loss or congestion can affect playback. Check network stats (RTT, packet loss) during playback.
Recommended diagnostic steps (in order)
1. Reproduce and correlate: Note timestamps when cut-outs occur and check agent logs for playback_finished, interruption events, VAD triggers, or TTS streaming errors.
2. Disable interruptions: Set allow_interruptions=False and retest.
3. Disable preemptive generation: Turn off preemptive_generation and test.
4. Test VAD sensitivity: Prewarm the VAD, lower sensitivity, or temporarily disable VAD to isolate the cause.
5. Test TTS provider/network: Try a different TTS provider or run on a known-good network to rule out streaming/latency issues.
6. Check observability: If you have Agent Observability, use traces and transcripts to pinpoint timing of interrupts and playback events.
If you’d like, tell me:
• Which TTS provider you’re using
• Whether you’re using Silero VAD and allow_interruptions or preemptive_generation settings
• Any relevant log snippets or timestamps when cut-outs happen
I can then suggest exact settings to change or specific log lines to look for.
Sources: Uninterruptable Agent | LiveKit docs | TTS Translator with Gladia STT | LiveKit docs | examples/voice_agents | livekit/livekit_composite | DeepWiki | Live Conversations with AI using ChatGPT and WebRTC | Resolving "InvalidState - failed to capture frame" errors in AudioSourcetall-airline-16756
01/02/2026, 2:31 PMallow_interruptions=False, preemptive_generation=False, and discard_audio_if_uninterruptible=True at the session level.
Targeted checks (quick, in this order)
1. Verify VAD false positives
◦ Check VAD event logs during playback for any VAD-triggered events. Even with allow_interruptions=False, a noisy/too-sensitive VAD can cause upstream behavior. Prewarm the Silero VAD and test with quiet background to rule this out.
1. Confirm TTS streaming stability
◦ Since use_streaming=True, check network stability and TTS streaming logs for partial/frame errors or reconnects. Test with use_streaming=False (non-streaming TTS) to see if cut-outs stop.
1. Validate discard behavior
◦ discard_audio_if_uninterruptible=True can drop audio when a generation is cancelled. Re-run with discard_audio_if_uninterruptible=False to see if audio continues instead of being discarded.
1. Inspect interplay between session and agent flags
◦ You set allow_interruptions=False on the agent but discard_audio_if_uninterruptible=True and preemptive_generation=False on session. Confirm the effective runtime flags (agent vs session) are applied the way you expect and not overridden.
1. Reproduce with simplified pipeline
◦ Temporarily swap STT to a passive stub (no VAD/STT input) and run only TTS playback to confirm whether the issue is TTS or something reacting to live input.
1. Check agent logs for playback lifecycle
◦ Look for playback_finished, playback_cancelled, or TTS plugin error lines with timestamps that match cut-outs.
Suggested quick experiments
• Experiment A: Set use_streaming=False in Google TTS and retest.
• Experiment B: Keep streaming, but set discard_audio_if_uninterruptible=False and ensure VAD is prewarmed.
• Experiment C: Temporarily replace Silero VAD with disabled VAD or higher threshold to rule out false positives.
If you want, share one log snippet covering the timestamps of a cut-out (VAD events + TTS streaming events). With that I can point to the exact event causing the interruption and suggest the exact flag or config change.tall-airline-16756
01/02/2026, 3:10 PMdef prewarm(proc: JobProcess):
proc.userdata["vad"] = silero.VAD.load()
server.setup_fnc = prewarm
1. In your RTC/session entrypoint, retrieve and pass the prewarmed VAD to the agent/session:
vad_instance = ctx.proc.userdata["vad"]
session = AgentSession(
...,
vad=vad_instance,
allow_interruptions=False,
)
Notes
• Prewarming avoids per-session model load latency and reduces false starts from VAD initialization.
• After you prewarm, confirm the agent is actually using vad_instance (not loading a new one) and check VAD logs for expected ready/heartbeat messages. Sources: Repeater | LiveKit docs | VAD Metrics | LiveKit docs | Uninterruptable Agent | LiveKit docsrich-keyboard-96538
01/02/2026, 3:11 PMtall-airline-16756
01/02/2026, 3:11 PM# example_agent.py
# Adapt imports to your project layout / package names
from livekit_agent import server, JobProcess, AgentSession # pseudo-imports — replace with your actual modules
import silero
import soniox
from google import LLM as GoogleLLM, TTS as GoogleTTS
# from sarvam import TTS as SarvamTTS # optional alternative TTS
# Prewarm function runs once per process before sessions start
def prewarm(proc: JobProcess):
# Load Silero VAD once and store on proc.userdata for reuse
proc.userdata["vad"] = silero.VAD.load()
# Optional: log that VAD is prewarmed
print("Silero VAD prewarmed and stored in proc.userdata['vad']")
# Assign the prewarm function to the server setup hook
server.setup_fnc = prewarm
# Example handler/entrypoint for creating an AgentSession per connection
def handle_new_session(ctx):
# Retrieve the prewarmed VAD instance
vad_instance = ctx.proc.userdata.get("vad")
if vad_instance is None:
# Fallback: load here (avoid in production; prewarming should have run)
vad_instance = silero.VAD.load()
# Configure components matching your provided setup
llm = GoogleLLM(
model="gemini-2.5-flash",
temperature=0.7,
max_output_tokens=200,
)
stt = soniox.STT(
params=soniox.STTOptions(
language_hints=["en", "hi", "ta", "te", "ml", "kn"],
context="User is talking on the phone with a customer support agent.",
enable_language_identification=True,
language_hints_strict=True,
)
)
tts = GoogleTTS(
language="en-IN",
voice_name="en-IN-Chirp3-HD-Aoede",
volume_gain_db=2.0,
use_streaming=True,
pitch=0,
# location="us-central1",
)
# Create the Agent (your agent class/init may differ)
agent = YourAgentClass(
instructions="Your agent instructions here",
vad=vad_instance,
turn_detection=MultilingualModel(), # replace with your actual turn detection init
llm=llm,
tts=tts,
stt=stt,
allow_interruptions=False,
)
# Create the AgentSession; session-level flags as you specified
session = AgentSession(
agent=agent, # if your API expects agent instance here; otherwise adapt
discard_audio_if_uninterruptible=True,
preemptive_generation=False,
)
# Start or run the session — adapt to your framework's session run method
session.run(ctx) # replace with actual run/serve call
# Wire the new-session handler to the server's connection hook
server.on_new_session(handle_new_session)
# If running as a script
if __name__ == "__main__":
# Start the server/process loop — replace with your actual server start
server.start()
Notes and small checks
• Replace placeholder imports (livekit_agent, YourAgentClass, MultilingualModel, server.start(), etc.) with the actual classes/functions in your codebase.
• Ensure server.setup_fnc = prewarm is set before the server/process starts so the VAD is loaded once per process.
• Confirm the session creation path uses the vad_instance from ctx.proc.userdata rather than calling silero.VAD.load() again.
• After deploying, verify prewarm by checking process logs for "Silero VAD prewarmed..." and confirm sessions use the same instance.
If you want, I can adapt this to the exact LiveKit SDK class names you’re using — share the module/class names for your Agent, AgentSession, and server if they differ.rich-keyboard-96538
01/02/2026, 5:21 PMtall-airline-16756
01/02/2026, 5:22 PMsip.callStatus and sip.callID
• Tracks whether active was ever seen for a call to distinguish hangup after answer vs hangup without answer
Tell me: do you want a full Python snippet now, and are you using LiveKit Agents with SIP participants?
Sources: SIP Lifecycle Management Agent | LiveKit docs | Outbound Calling Script | LiveKit docsrich-keyboard-96538
01/02/2026, 5:23 PMtall-airline-16756
01/02/2026, 5:23 PMsip.callStatus, and classifies each call as answered, rejected, or no-answer (missed after ringing). Adjust imports and integration points to match your project's LiveKit client/server library.
import asyncio
from collections import defaultdict
from datetime import datetime, timezone
# Replace these with your actual LiveKit SDK imports
# from livekit import Room, Participant, ParticipantKind
# Simple call tracker to determine outcomes
class CallTracker:
def __init__(self):
# call_id -> {seen_active: bool, first_seen: datetime, last_status: str}
self.calls = defaultdict(lambda: {"seen_active": False, "first_seen": None, "last_status": None})
def update_status(self, call_id: str, new_status: str):
now = datetime.now(timezone.utc)
entry = self.calls[call_id]
if entry["first_seen"] is None:
entry["first_seen"] = now
entry["last_status"] = new_status
if new_status == "active":
entry["seen_active"] = True
def finalize_on_hangup(self, call_id: str):
entry = self.calls.get(call_id)
if not entry:
return None
seen_active = entry["seen_active"]
last_status = entry["last_status"]
if last_status == "hangup" and not seen_active:
return "no_answer" # rang and then hung up without ever being active
if last_status == "hangup" and seen_active:
return "answered_then_hungup"
if last_status == "active":
return "answered"
return last_status
# Instantiate a tracker
tracker = CallTracker()
# Example event handlers (adapt to your SDK event wiring)
async def on_participant_connected(participant):
# Only handle SIP participants
if participant.kind != "sip": # replace with ParticipantKind.PARTICIPANT_KIND_SIP if available
return
call_id = participant.attributes.get("sip.callID") or participant.sid or participant.identity
call_status = participant.attributes.get("sip.callStatus", "unknown")
tracker.update_status(call_id, call_status)
print(f"[connected] call_id={call_id} status={call_status}")
if call_status == "active":
print(f"✅ Call answered (participant connected active): {call_id}")
if call_status == "hangup":
outcome = tracker.finalize_on_hangup(call_id)
print(f"📴 Call hangup detected on connect: {call_id} -> outcome={outcome}")
# Listen to attribute changes for live updates
async def on_participant_attributes_changed(participant, changed_attributes: dict):
# Only handle SIP participants
if participant.kind != "sip":
return
call_id = participant.attributes.get("sip.callID") or participant.sid or participant.identity
if "sip.callStatus" in changed_attributes:
new_status = changed_attributes["sip.callStatus"]
tracker.update_status(call_id, new_status)
print(f"[attr_change] call_id={call_id} sip.callStatus={new_status}")
if new_status == "active":
print(f"✅ User has answered the call: {call_id}")
if new_status == "hangup":
outcome = tracker.finalize_on_hangup(call_id)
if outcome == "no_answer":
print(f"💤 Missed / no answer (ringed then hung up): {call_id}")
elif outcome == "answered_then_hungup":
print(f"📞 Answered then hung up: {call_id}")
else:
print(f"📴 Hangup outcome: {call_id} -> {outcome}")
# Optional: participant disconnected or removed handler
async def on_participant_disconnected(participant):
if participant.kind != "sip":
return
call_id = participant.attributes.get("sip.callID") or participant.sid or participant.identity
# If participant disconnects without explicit hangup attribute, treat as hangup
if tracker.calls.get(call_id):
tracker.update_status(call_id, "hangup")
outcome = tracker.finalize_on_hangup(call_id)
print(f"[disconnected] call_id={call_id} -> outcome={outcome}")
# Example wiring function to attach handlers to your LiveKit Room object
def wire_handlers(room):
# Replace these with your SDK's event registration methods
room.on("participant_connected", lambda p: asyncio.create_task(on_participant_connected(p)))
room.on("participant_updated", lambda p, changes: asyncio.create_task(on_participant_attributes_changed(p, changes)))
room.on("participant_disconnected", lambda p: asyncio.create_task(on_participant_disconnected(p)))
# Usage example (pseudo-start)
if __name__ == "__main__":
# room = Room(...) # create/connect to your LiveKit room
# wire_handlers(room)
# run your event loop / room connect lifecycle
pass
How the code decides outcomes
• Answered: sip.callStatus becomes "active".
• Rejected: final sip.callStatus becomes "hangup" but the tracker had previously seen "active" → treated as answered then hung up (rejected during ringing would also show hangup without active).
• No-answer (missed): final sip.callStatus == "hangup" and active was never seen for that call ID.
If you want, I can adapt this to the exact LiveKit Python SDK classes/method names you use—share the SDK import names and event APIs and I’ll match them.