To reduce the 2-3 minute latency in your TTS and STT model combination, consider the following optimizations: • Optimize Network Proximity: Ensure your agent is close to the STT and TTS services to minimize network latency. • Configure STT for Lower Latency: Use Deepgram's STT with settings like

noDelay=True

for speed and

interimResults=True

for partial transcriptions. • Enable Preemptive Generation: Set

preemptive_generation=True

in your configuration to start TTS generation before the user finishes speaking. • Monitor Metrics: Use metrics like Time To First Token (TTFT) and SpeechLatency to identify bottlenecks. • Low-Latency TTS: Consider using a TTS model optimized for low latency, such as OpenAI's TTS services. Here’s an example configuration for Deepgram STT:

session = AgentSession(
    stt=deepgram.STT(
        model="nova-3",
        language="en-US",
        noDelay=True,
        interimResults=True
    ),
    tts=openai.TTS(),
    preemptive_generation=True
)

These adjustments should help reduce latency. Let me know if you need further assistance! Sources: How can I reduce latency in voice agents using STT, TTS and LLM? | plugins/deepgram/src/stt.ts | AI Agents Framework | livekit/livekit_composite | DeepWiki | STTOptions | LiveKit Agents