wooden-librarian-60399
12/22/2025, 4:57 PMtall-airline-16756
12/22/2025, 4:58 PMtall-airline-16756
12/22/2025, 4:58 PMuser_state_changed when speech begins.
1. STT streaming and interim transcripts
◦ Speech-to-text produces interim transcripts and emits user_input_transcribed for non-final partial transcripts.
1. Turn detection / endpointing
◦ The system detects turn end via VAD, STT endpointing, realtime model, or manual session.commit_user_turn(). Endpointing uses min_endpointing_delay and max_endpointing_delay.
1. Final transcript event
◦ A final transcript is produced and user_input_transcribed is emitted with is_final=true.
1. on_user_turn_completed hook
◦ on_user_turn_completed runs before agent response; you can perform RAG lookups, modify/cancel the message (e.g., call StopResponse()), or inject context.
1. Conversation item added
◦ The user message is appended to conversation history and conversation_item_added is emitted.
1. Agent state changes to “thinking”
◦ Agent transitions to thinking and emits agent_state_changed with new_state="thinking".
After these steps the agent begins generating the response with the LLM.
If you want, I can:
• Show where to customize on_user_turn_completed, or
• Map these events to the specific SDK hooks you’re using (JS, Python, or server). Which would help you most? Sources: Turns overview | LiveKit docs | Agent session | LiveKit docs | Interrupt User | LiveKit docs | Pipeline nodes and hooks | LiveKit docs | Agent speech and audio | LiveKit docswooden-librarian-60399
12/22/2025, 5:00 PMtall-airline-16756
12/22/2025, 5:00 PMuser_state_changed ("speaking") and interim user_input_transcribed events emitted. These interim transcripts can trigger preemptive generation immediately if preemptive_generation is enabled.
2. STT streaming produces interim transcripts while preemptive LLM work runs in parallel.
3. Turn detection / endpointing determines the end of the user turn (VAD, STT endpointing, realtime model, or session.commit_user_turn()).
4. When a final transcript is produced -> user_input_transcribed with is_final=true. The system compares the final transcript to the interim one used for preemptive generation:
◦ If they match closely, the preemptive response is reused to reduce latency.
◦ If they differ significantly, the preemptive result is discarded and a fresh LLM inference is started with the final transcript.
1. on_user_turn_completed runs after final transcript and before agent reply. If this hook modifies context (RAG lookups, message edits, or cancels the reply via StopResponse()), any preemptive output is invalidated and a new response is generated reflecting the updated context.
2. Conversation item is added (conversation_item_added) and agent state changes to thinking. If preemptive output was valid it proceeds; otherwise a new LLM call runs, then TTS and playback.
Interruption and cancellation
• If the user interrupts mid-speech and interruption criteria are met (e.g., min_interruption_duration), the pipeline cancels queued/ongoing speech and preemptive responses and restarts inference with the new input.
Practical implications
• Preemptive generation trades off latency vs. correctness: it speeds replies when interim transcripts are accurate, but is always verified against final transcript and context changes.
• Any server-side changes in on_user_turn_completed will force regeneration to ensure accuracy.
Would you like me to show where to enable/configure preemptive_generation and interruption thresholds in your SDK (JS or Python), or an example of handling on_user_turn_completed?
Sources: examples/voice_agents | livekit/livekit_composite | DeepWiki | Agent speech and audio | LiveKit docs | Pipeline nodes and hooks | LiveKit docswooden-librarian-60399
12/22/2025, 5:03 PMtall-airline-16756
12/22/2025, 5:03 PMagent_started_processing.
Use these instead to detect when processing begins:
• agent_state_changed with new_state = "thinking" — definitive signal that the agent has started LLM inference.
• conversation_item_added — indicates the LLM has begun generating content (useful for detecting generation start).
• Preemptive generation may cause an earlier transition to thinking (or earlier generation events) while the user is still speaking.
Would you like a short SDK example (JS or Python) to listen for agent_state_changed and conversation_item_added?
Sources: Agent session | LiveKit docs | Turns overview | LiveKit docs | examples/voice_agents | livekit/livekit_composite | DeepWiki | Agent speech and audio | LiveKit docs | Agent Builder | LiveKit docs