To achieve your goal of a configurable LiveKit agent that accepts custom tools, integrates with a RAG (retrieval-augmented generation) tool, initializes system instructions dynamically, and conditionally performs knowledge base lookups, you can follow the architectural approach outlined below: 1. Configurable Architecture: Use a configuration object (

AgentConfig

) to manage agent instructions, RAG settings, and custom tools. This allows you to easily enable or disable RAG, specify index/data paths, and inject toolsets. 2. Custom Tools: Accept a list of custom tools (decorated as function tools) and register them so the agent has callable utility functions at runtime. 3. RAG Tool Integration: Instantiate a RAG handler (e.g.,

RAGHandler

) only if enabled via the configuration, and use it to enrich the session with retrieved knowledge base content where appropriate. 4. Dynamic Instructions: Add a method (e.g.,

update_instructions_dynamically()

) that updates system instructions each session by combining static and contextual content. 5. Conditional RAG Calls: Implement logic such as

should_perform_rag_lookup()

that checks whether user input indicates a need for knowledge search (e.g., by scanning for question patterns) and executes the RAG retrieval handler only when relevant. Reference Implementation Snippet:

class ConfigurableRAGAgent(Agent):
    def __init__(self, config: AgentConfig):
        super().__init__(instructions=config.instructions)
        ...
        if config.enable_rag: ... initialize RAGHandler ...
        if config.custom_tools: ... register tools ...

    async def on_user_turn_completed(self, turn_ctx, new_message):
        ...
        if await self.should_perform_rag_lookup(user_input):
            ... retrieve and inject context ...

This modular approach ensures the agent can be initialized dynamically and supports both advanced RAG-assisted and regular conversation capabilities. For detailed Python code and further explanation, let me know which part you'd like to focus on or if you need the full implementation template. Sources: rag/README.md | Tool definition and use | LiveKit Docs | External data and RAG | LiveKit Docs