Thanks — this is a great question. I’m responding to your note that about 1 in 10 times the agent switches turn but the user’s voice is not detected. Below I list the most likely causes and practical steps you can take to diagnose and reduce those missed user turns. Likely causes • Noise cancellation suppressing speech Noise filters (e.g., telephony noise reduction or krisp-like processors) can remove low-volume speech. The VAD may still flip a turn based on raw activity but the STT then receives suppressed audio and returns no words. • Very short or low-confidence utterances Brief replies (1–3 words) often fail STT confidence thresholds and end up as “no transcription” even though a turn switch occurred. • VAD/end-of-turn fires too early on a pause VAD detects silence and signals end-of-turn while the user is still in a natural pause. The agent begins responding and subsequent user speech is ignored or arrives outside the expected window. • False interruption or transient noise spike A short noise burst or glitch can trigger the agent to switch state, but no valid speech follows. The system then times out without a transcription. • Mismatch or routing/processing gap between VAD and STT VAD and STT must use the same processed audio stream. If routing or preprocessing differs (volume/gain, filters), STT may get audio that’s too quiet or clipped. • Latency / streaming gaps Network jitter or buffering can drop the beginning of the user’s utterance so STT never sees transcribable audio. • Agent configured as uninterruptible If interruptions are disabled, any user speech after the agent starts responding may be discarded rather than reprocessed. Suggested diagnostic steps 1. Reproduce with logs enabled ◦ Log the exact state transitions (e.g., user → listening, agent → speaking). ◦ Log STT events: when transcription starts, returns empty result, or times out. ◦ Check for any agent false-interruption events. 1. Test without noise cancellation ◦ Temporarily disable noise reduction to see if detection improves. If so, your noise suppression is cutting valid speech. 1. Try increasing audio gain for testing ◦ Amplify incoming audio to confirm whether volume is the root cause. 1. Evaluate short-utterance behavior ◦ Run tests with short answers (single words) to see STT success rate and adjust STT confidence/endpointing thresholds if needed. 1. Use STT- or model-based turn detection instead of pure VAD ◦ Semantic turn detectors (end-of-turn models) are better with natural pauses and reduce premature switches. 1. Enable interruption handling (if appropriate) ◦ Set allow_interruptions=True so a user can cut in if the agent starts responding prematurely. Also monitor false_interruption_timeout behavior. 1. Verify end-to-end audio routing and preprocessing ◦ Ensure the same processed stream feeds VAD and STT (same filters, gain, and routing). 1. Monitor for network jitter and buffering issues ◦ Check audio timestamps and packet loss during problem sessions. Quick configuration ideas to try • Temporarily disable noise cancellation and run a batch of calls. • Switch from VAD-only endpointing to STT endpointing or a Multilingual/semantic turn detection model. • Allow interruptions during agent speech while you iterate on false-interruption timeouts. Short summary The most common root causes are premature VAD end-of-turn (often triggered by pauses or noise) combined with short/low-volume user replies and/or aggressive noise cancellation. Moving to a semantic turn detection model, ensuring consistent audio routing to STT, enabling interruptions during agent speech for testing, and temporarily disabling noise cancellation will usually surface which factor is dominant and reduce the 10% failure rate. If you can share any logs showing the state transitions (user/agent) and STT results for one or two problem calls, I can help interpret them and recommend targeted parameter adjustments. Sources: Troubleshooting STT not picking up utterances | LiveKit | How does end-of-utterance detection work in conversations? | Turns overview | LiveKit Documentation | AssemblyAI plugin guide | LiveKit Documentation | Improved End-of-Turn Model Cuts Voice AI Interruptions 39%