You're asking how transcription confidence is used in LiveKit and when transcripts are withheld from the LLM and TTS. Here's a concise, actionable summary. How confidence works • STT engines return interim (partial) and final transcripts plus confidence metadata. • LiveKit uses an end-of-utterance / end-of-turn confidence threshold (e.g., AssemblyAI’s default ~0.4) to decide whether to finalize a turn. Higher confidence triggers faster finalization; low confidence can delay endpointing. • Finalized transcripts are marked

is_final: true

and considered stable for downstream processing. When transcripts are NOT passed to the LLM or TTS • Interim (non-final) transcripts are never forwarded to LLM or TTS. • If end-of-turn isn’t detected (VAD or confidence model indicates user may continue), the transcript is not committed and not forwarded. • With

turn_detection = "manual"

, transcripts are held until

commit_user_turn()

is called. • Realtime model configurations that suppress forwarding (e.g., certain Realtime LLM setups or modalities that isolate TTS) can prevent transcripts from being sent. • If conversation history or model routing causes the system to respond in text only, TTS may not be invoked even after a final transcript. Important consequence • Only final, committed transcripts that meet endpointing/confidence rules are sent to the LLM and then to TTS. If you’d like, tell me: • which STT provider and turn detection mode you’re using (e.g., AssemblyAI + automatic or manual), and I’ll give exact settings and values to tune endpointing and forwarding. Sources: AssemblyAI plugin guide | LiveKit Docs | Events and error handling | LiveKit docs | How does end-of-utterance detection work in conversations? | examples/voice_agents | livekit/livekit_composite | DeepWiki | Azure OpenAI Realtime API and LiveKit | LiveKit docs