Thanks — I’m glad you flagged this. You said the LiveKit Community Bot takes about 5 seconds to respond when using OpenRouter; below I’ll explain the likely causes and give practical steps you can try to reduce that latency. Why you’re seeing ~5s • The delay is usually the sum of several pipeline stages: Speech-to-Text (if you use STT), the Large Language Model’s Time To First Token (TTFT), Text-to-Speech (TTS), and network round-trips between your agent, OpenRouter, and LiveKit. • TTFT (time from request to the first LLM token) is often the single largest contributor. • Network proximity (where your backend/agent runs relative to OpenRouter and LiveKit) can add noticeable round-trip time. Practical steps to reduce response time 1. Deploy your agent closer to the model provider ◦ Host your agent/backend in a cloud region near OpenRouter’s servers to reduce network RTT. 1. Use streaming end-to-end ◦ Ensure STT, LLM, and TTS are all configured for streaming so partial data is processed and returned incrementally rather than waiting for full utterances. 1. Pick lower-latency models and optimize OpenRouter settings ◦ Use faster/lightweight models available via OpenRouter (or

openrouter/auto

routing) for interactive use. ◦ Configure

provider

and

fallback_models

to prefer low-latency providers/models. 1. Measure and monitor specific metrics ◦ Track TTFT and TTFB to identify which stage (STT, LLM, or TTS) is the biggest bottleneck. ◦ Use observability/tracing on the agent to see per-stage durations. 1. Consider LiveKit Inference (if relevant) ◦ LiveKit Inference provides global co-location and dynamic routing to reduce latency by placing inference near where traffic originates. If you want, I can help further — please tell me which of these apply so I can give targeted steps: • Are you running STT and TTS in your pipeline, or just text-based prompts? • Where is your agent hosted (region/cloud provider)? • Which OpenRouter model or provider are you using now? • Are you using streaming responses today? With those details I’ll suggest the next concrete changes you can make. Sources: examples/voice_agents | livekit/livekit_composite | DeepWiki | How can I reduce latency in voice agents using STT, TTS and LLM? | LiveKit | OpenRouter LLM plugin guide | LiveKit Documentation | Introducing LiveKit Inference: A unified model interface for voice AI