How to improve latency
# ask-aiq
quaint-afternoon-36516
08/01/2025, 6:55 AMHow to improve latency
t
tall-airline-16756
08/01/2025, 6:55 AMThanks for reaching out, Mihir. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
08/01/2025, 6:56 AMTo improve latency in LiveKit, consider the following strategies:
• Network Proximity Optimization: Place your services close to LiveKit servers to minimize latency. Use LiveKit Cloud's global mesh to connect users to the nearest server, similar to a CDN for real-time media.
• Monitor and Analyze Latency: Use the Agents Metrics API to track performance metrics like Time To First Token (TTFT) and Time To First Byte (TTFB). Implement logging in your SDK to debug latency issues.
• Infrastructure and Configuration: Regularly update LiveKit to the latest version for performance improvements. Enable dynamic playout delay for audio tracks and handle network congestion with custom UI feedback.
• Optimization Techniques: Reduce NACK traffic by delaying retransmissions and ensure audio-video synchronization for a smoother experience.
• Case Study Insights: Learn from examples like Decentraland, which significantly reduced latency by switching to LiveKit, and the Rover Teleop project, which achieved ~190ms latency.
These steps can help enhance real-time communication performance. Let me know if you need further details on any of these strategies.
Sources: Voice Assistants | livekit/livekit_composite | DeepWiki | Decentraland's Catalyst: using WebRTC to interact in the metaverse | CHANGELOG.md | README.md
q
quaint-afternoon-36516
08/01/2025, 6:57 AMsession = AgentSession(
llm=openai.LLM(model="gpt-4o-mini"),
stt=deepgram.STT(model="nova-3", language="multi"),
tts=deepgram.TTS(model="aura-2-andromeda-en"),
vad=ctx.proc.userdata["vad"],
_turn_detection_=MultilingualModel(),
_preemptive_generation_=False, # or True, if optimizing latency
)
this combination of models for tts, stt has around 2-3 minute , how i can improve?
t
tall-airline-16756
08/01/2025, 6:57 AMHey there! To help speed up the latency of your TTS and STT models in LiveKit, here are some friendly tips you might find useful:
• Optimize Network Proximity: Try to have your agent located close to the LLM, STT, and TTS services. This can really help cut down on network delays.
• Use Streaming APIs: If your STT and TTS providers offer streaming APIs, switching to those can make things faster compared to the usual non-streaming methods.
• Choose Low-Latency Models: For TTS, models like Rime are great for real-time use. For STT, smaller models like Whisper (especially if you run them locally) can reduce latency, though keep in mind they might support fewer languages.
• Optimize Audio Processing: Adjusting audio frame sizes—like using a 20ms frame size with WebRTC’s default encoding—can help balance speed and accuracy.
• Monitor and Analyze Metrics: Keep an eye on latency using metrics APIs, focusing on Time To First Token (TTFT) and Time To First Byte (TTFB) to spot any slow spots.
These ideas should help bring down that 2-3 minute delay you’re seeing. If you want to dive deeper into any of these or need a hand, just let me know—I’m here to help!
Sources: Live Conversations with AI using ChatGPT and WebRTC
6 Views