To reduce latency and enable real-time streaming between your components (STT, VAD, TTS, and LLM), consider the following strategies: • Network Proximity: Ensure your agent is network-close to your LLM, STT, and TTS services. Deploying your agent in the same region as these services can significantly reduce round-trip times. • Preemptive Generation: Enable

preemptive_generation

in your

AgentSession

. This allows the agent to start generating responses as soon as the final transcript is available, even before the user's turn is fully committed. This is especially effective if your STT returns results faster than VAD emits

end_of_speech

events. • Optimize AgentSession Timing: Adjust parameters such as

min_interruption_duration

,

min_endpointing_delay

, and

max_endpointing_delay

to fine-tune how quickly the agent responds after detecting end-of-utterance signals. • Streaming Adapters: If your STT or TTS providers do not natively support streaming, use streaming adapters to enable real-time data flow between components. • Turn Detection Modes: Use the most responsive turn detection mode available. The session will prioritize

realtime_llm

→

vad

→

stt

→

manual

for optimal latency. • Monitor Latency: Use the Agents metrics API to track Time To First Token (TTFT) and Time To First Byte (TTFB) to identify bottlenecks. By combining these strategies, you can achieve lower end-to-end latency and more responsive real-time streaming in your pipeline. Sources: Voice Assistants | livekit/livekit_composite | DeepWiki | Speech & Audio Processing | livekit/livekit_composite | DeepWiki | Agent speech and audio | LiveKit Docs | livekit-agents/livekit/agents/voice/agent_session.py | agents/src/pipeline/pipeline_agent.ts