Thanks — you're asking how to attach a large knowledge base to a voice agent while keeping lag low. Below are practical options and recommendations so you can pick the approach that fits your needs. Which approach to use • Use RAG with on_user_turn_completed when you need fast, contextual lookups injected into the conversation. This runs the retrieval alongside LLM processing and avoids an extra tool-call round trip (lowest latency in STT→LLM→TTS pipelines). • Use tool calls (structured function calls like

search_knowledge_base

) for precise, explicit queries or when you must control retrieval logic and provenance. This is slightly higher latency than RAG because it adds a round trip. Latency-reduction best practices 1. Co-locate services — run the agent, STT, LLM, and retrieval service in the same region to cut network RTTs. 2. Use a streaming pipeline (stream STT → LLM → TTS) so stages overlap instead of running sequentially. 3. Cache frequent answers and pre-cache TTS for canned responses to avoid repeated compute and synthesis. 4. Use faster/smaller models for quick replies and reserve larger models for complex queries. 5. Optimize retrieval: ◦ Shard or index your KB for targeted searches. ◦ Use embeddings + approximate nearest neighbor (ANN) search for fast vector lookups. ◦ Limit retrieved context size to only relevant snippets. 1. Provide user feedback during longer lookups: ◦ Short spoken prompts (e.g., “Checking that now”) or pre-synthesized audio. ◦ Ambient “thinking” sound where appropriate. 1. Monitor and iterate: ◦ Track TTFT, TTFB, and end-to-end latency. ◦ Identify whether STT, retrieval, LLM, or TTS is the bottleneck and optimize that layer. Quick deployment guidance 1. If you need the lowest latency and are using STT+LLM+TTS pipeline → start with RAG via

on_user_turn_completed

. 2. If you need strict control or provenance → implement tool calls but optimize to reduce round trips (cache, parallelize retrieval). 3. Always colocate and enable streaming; pre-cache TTS for fixed phrases. If you want, tell me: • Which pipeline you’re using (STT+LLM+TTS or something else), • The KB size and type (documents, FAQ, embeddings), and • Any latency targets (e.g., <1s end-to-end), and I’ll recommend a concrete architecture and retrieval/settings to implement. Sources: External data and RAG | LiveKit Documentation | Understand and Improve Agent Latency | LiveKit | Voice Agent Architecture: STT, LLM, and TTS Pipelines Explained | LiveKit | How can I reduce latency in voice agents using STT, TTS and LLM? | Voice Agent Architecture: STT, LLM, and TTS Pipelines Explained | LiveKit, with interactive elements