I'm glad you're looking into ways to reduce perceived latency in LiveKit! Here are some friendly tips that might help you along the way: • Preemptive Speech Generation: You can enable agents to start generating responses even before the user finishes their turn by using partial transcription or early input signals. This works great in Python, especially with fast STT (Speech-to-Text) systems. • Preconnect Buffer for Microphone Tracks: Consider using the

preConnectBuffer

option to locally buffer audio before the connection is fully set up. This can really help reduce latency when users start speaking early. • TTS Latency Reduction: If you're using Rime for text-to-speech, setting the

reduce_latency

parameter to

true

speeds up TTS inference by skipping text normalization—handy for quicker responses! • Infrastructure Optimization: Deploying agents close to your LLM, STT, and TTS services can minimize network delays. Plus, leveraging LiveKit Cloud’s global edge network ensures low-latency transmission, which is fantastic for real-time interactions. • Startup Efficiency: Pre-downloading models and using dependency layer caching can really help cut down startup latency, making everything feel snappier. • Performance Monitoring: Don’t forget to use LiveKit’s metrics API to keep an eye on latency patterns and spot any bottlenecks early on. • Streaming Optimization: Minimizing delays at every step—from data reception to processing and transmission—helps maintain a smooth and seamless experience for everyone involved. These approaches cover both the technical and architectural sides of things, aiming to boost your real-time performance. If you’d like, I’m here to help you dive deeper into any of these strategies! Sources: Rime integration guide | LiveKit Docs | Rime integration guide | LiveKit Docs | Agents Overview | LiveKit Docs | Live Conversations with AI using ChatGPT and WebRTC | Live Conversations with AI using ChatGPT and WebRTC

Thanks for reaching out! Could you please clarify what you mean by using filler words to reduce perceived latency? Are you hoping to programmatically insert filler words during speech synthesis, or is it related to another part of the process? Sharing a bit more about your use case would really help me support you better.

I'm glad you're exploring how to programmatically insert filler words during speech synthesis in LiveKit! You can achieve this by using the

tts_node

callback within a custom

Agent

subclass. This handy approach lets you tweak the text stream before it reaches the TTS engine, allowing you to add natural-sounding phrases like "um," "uh," or "let me think" to make the speech feel more conversational and human-like. Here's a simple example to get you started:

async def tts_node(
    self,
    text: AsyncIterable[str],
    model_settings: ModelSettings
) -> AsyncIterable[rtc.AudioFrame]:
    """
    Prepend a common hesitation phrase to the agent's full response.
    """
    async def add_filler(input_text: AsyncIterable[str]) -> AsyncIterable[str]:
        yield "um... let me think "

        async for chunk in input_text:
            yield chunk

    return await Agent.default.tts_node(self, add_filler(text), model_settings)

This method gently adds a filler phrase right at the beginning of the agent's response, just before synthesis happens. For even more dynamic interactions, you might want to explore the

on_user_turn_completed

lifecycle hook. It lets you generate a quick filler response using a fast LLM—something like "Let me think about that..."—while the main answer is still being prepared. Playing this filler right away can really enhance how responsive the system feels. If you'd like, I’d be happy to share more details or tailor examples for specific scenarios, such as inserting fillers conditionally or integrating with a particular TTS provider. Just let me know how I can assist you further! Sources: Agents 0.x migration guide | LiveKit Docs | Agents v0.x migration guide - Node.js | LiveKit Docs | livekit-plugins/livekit-plugins-deepgram/livekit/plugins/deepgram/stt.py | Docs::Agents | pipeline-llm/transcription_node.py

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!