Hi — welcome to LiveKit! 👋 Thanks for reaching out. How can I help you today? Here are a few things I can assist with—pick any that match your need or tell me something else: • Troubleshooting a connection, join/leave, or media quality issue • SDK setup or sample code (Web, iOS, Android) • Room, participant, or permission questions • Token/authentication guidance • Billing, account, or plan questions • Feature questions or integration help If you can, please share: • The SDK/platform you’re using (Web/iOS/Android/Server) • Any error messages, room ID, or request IDs • A brief description of the problem or goal I’ll get started as soon as you send a bit more detail. , with interactive elements

Thanks — I’m responding to your question about the best open-source LLM for low-latency use with LiveKit. Short recommendation • For the best balance of low latency and quality: use a smaller, efficient model such as Llama 3 (8B) or Mistral 7B. • For the absolute lowest latency and on-prem control: run a lightweight model locally via Ollama (or another local runner). • If you want managed hosting with low-latency endpoints: use providers like Together AI, Cerebras, or Groq that host these open models and integrate via LiveKit plugins. Why these choices • Smaller models (7B–8B) have much lower inference latency than 30B+ or 100B models while still giving good conversational quality for many real‑time voice/agent scenarios. • Local inference (Ollama) removes network roundtrips and gives predictable latency, great for real‑time use. • Managed providers (Together, Cerebras, Groq) offer optimized inference infra and streaming support, reducing end‑to‑end delay without requiring you to host hardware. Integration options with LiveKit • LiveKit Inference: access some models through a unified LiveKit API key for simplified setup (good if a supported model is available). • Plugins/direct provider integration: configure provider plugins (Together AI, Ollama, Cerebras, Groq, Baseten, etc.) for more model choices or self‑hosted setups. Latency best practices 1. Use streaming end‑to‑end (STT → LLM → TTS) so partial results flow immediately. 2. Co-locate your agent and the model provider where possible to reduce network hops. 3. Prefer smaller distilled models (7B–8B) for real‑time interactions. 4. If you self‑host, use optimized runtimes (quantized weights, GPU/TPU inference, or inference accelerators like Groq/Cerebras). 5. Benchmark models in your exact pipeline (audio length, concurrency) — real-world latency varies by setup. Suggested starting points • Llama 3.1 8B via Together AI — good latency/quality tradeoff. • Mistral 7B via Ollama (local) — very low latency when run on local GPU. • DeepSeek or Cerebras-hosted Qwen if you prefer managed options available through LiveKit Inference. If you’d like, I can help next with: • A suggested test plan to benchmark latency (what to measure, how to measure). • Example LiveKit plugin configurations for Together AI or Ollama. • Guidance on streaming setup (STT → LLM → TTS) for minimum end‑to‑end latency. Which option interests you most — local Ollama, a managed provider (Together/Cerebras/Groq), or LiveKit Inference? Sources: DeepSeek LLM | LiveKit Documentation | Cerebras and LiveKit | LiveKit Documentation | Large language models (LLM) overview | LiveKit Documentation | Ollama LLM plugin guide | LiveKit Documentation, with interactive elements