How to improve latency for an voice agent?

Thanks for reaching out, Yashish. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

can you provide a python code for that?

Hey there! Let me walk you through a super cool Python code example for setting up a LiveKit voice agent session. This setup is designed to be lightning-fast and super responsive, just like chatting with a quick-witted friend.

import logging
from dotenv import load_dotenv
from livekit.agents import Agent, AgentSession, JobContext, JobProcess, WorkerOptions, cli, metrics
from livekit.agents.llm import ChatContext, ChatMessage
from livekit.agents.voice import AgentStateChangedEvent, MetricsCollectedEvent
from livekit.plugins import deepgram, openai, silero
from livekit.plugins.turn_detector.multilingual import MultilingualModel

load_dotenv()
logger = logging.getLogger("preemptive-generation")

class MyAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="Your name is Kelly. You would interact with users via voice. "
            "With that in mind keep your responses concise and to the point. "
            "You are curious and friendly, and have a sense of humor."
        )

    async def on_enter(self):
        self.session.generate_reply(instructions="say hello to the user")

    async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage):
        # Update the chat context or tools here will cancel the preemptive generation
        pass

async def entrypoint(ctx: JobContext):
    session = AgentSession(
        vad=ctx.proc.userdata["vad"],
        llm=openai.LLM(model="gpt-4o-mini"),
        stt=deepgram.STT(model="nova-3", language="multi"),
        tts=openai.TTS(),
        turn_detection=MultilingualModel(),
        preemptive_generation=True,  # Enable preemptive generation
        min_endpointing_delay=0.5,   # Minimum delay for endpointing
        max_endpointing_delay=6.0,   # Maximum delay for endpointing
        allow_interruptions=True,    # Allow user interruptions
    )

    last_eou_metrics = None

    @session.on("metrics_collected")
    def _on_metrics_collected(ev: MetricsCollectedEvent):
        nonlocal last_eou_metrics
        metrics.log_metrics(ev.metrics)
        if ev.metrics.type == "eou_metrics":
            last_eou_metrics = ev.metrics

    @session.on("agent_state_changed")
    def _on_agent_state_changed(ev: AgentStateChangedEvent):
        if (
            ev.new_state == "speaking"
            and last_eou_metrics
            and last_eou_metrics.speech_id == session.current_speech.id
        ):
            <http://logger.info|logger.info>(
                f"End-to-end latency: {ev.created_at - last_eou_metrics.last_speaking_time}"
            )

    await session.start(agent=MyAgent(), room=ctx.room)

def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm))

I've got some awesome features packed into this code! We're talking preemptive generation (so quick!), super low delays, and the ability to jump in and interrupt. Plus, we're tracking metrics to make sure everything runs smoothly. Feel free to tweak the STT, TTS, and LLM providers to fit exactly what you need. Let me know if you want to dive deeper! Sources: examples/voice_agents/basic_agent.py | Agent speech and audio | LiveKit Docs

are there any more ways also that you can implement?

Hey there! I totally get how important smooth, quick interactions are for a voice agent. Let me share some awesome techniques that can help you slash those latency times and create an even more responsive experience. • Streaming TTS with Priority Management: Imagine starting audio playback almost instantly as the first chunks come in! You can even set up smart priority levels to smoothly switch between messages if something more urgent pops up. • Pre-connect Buffering: Think of this like warming up before a race - start processing audio right when you connect to cut down those initial waiting moments. • Fine-tuned Endpointing and Interruption Parameters: It's like fine-tuning a musical instrument. Adjust those

min_endpointing_delay

,

max_endpointing_delay

,

min_interruption_duration

settings to make your conversation flow super naturally. • Consecutive Speech Delay Management: Use

min_consecutive_speech_delay

to keep your conversations feeling smooth and not too choppy. It's all about that conversational rhythm! • Remote Processing Architecture: For those heavy-duty tasks, imagine having a dedicated teammate who jumps right into the conversation and handles complex processing - that's what this technique does! • Provider-Specific Optimizations: Shop around for the speediest providers and follow their insider tips. Every little optimization counts! • Comprehensive Metrics and Monitoring: Keep an eye on those performance metrics like Time To First Token and component-level latencies. It's like having a health tracker for your voice agent! By mixing these smart techniques with some proactive planning, you'll be well on your way to creating an incredibly responsive and delightful voice agent. You've got this! Sources: examples/voice_agents/preemptive_generation.py | Bringing AI avatars to voice agents | Agents Core Architecture | livekit/livekit_composite | DeepWiki | AI Agents Framework | livekit/livekit_composite | DeepWiki | Voice Assistants | livekit/livekit_composite | DeepWiki