https://livekit.io logo
Join Slack
Powered by
I am developing a multi-agent voice system using t...
# ask-ai
c
I am developing a multi-agent voice system using the LiveKit framework and are encountering a critical issue with chat history persistence when switching between agents. Our system frequently hands off tasks from one agent to another, including to an orchestrator agent that transfers tasks to sub-agents. Currently, when we switch from one agent to another, all function calls and their corresponding responses from the previous agent are lost from the chat history. While user and assistant messages are retained, the removal of function call data means new agents lack the necessary context to understand the preceding actions. For instance, if a "presenter" agent uses a start_demo tool, and the conversation is then transferred to a "Q&A" agent, the Q&A agent has no record of the demo having been started. I am using the chat_ctx for all the agents. Do. you have any idea of why it's doing that and how I could solve it? Do I have to add them manually each time? Why do they persist for a single agent but not multiples?
t
Thanks for reaching out, Clovis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Function call and response messages are lost from chat history during agent handoffs in LiveKit because, by default, the chat context copying and truncation methods filter out function calls unless explicitly configured to retain them. To ensure function call data persists across agent handoffs and new agents have full context, you should explicitly set the relevant parameters when copying chat context: • Set 
keep_function_call=True
when using `_truncate_chat_ctx`:
Copy code
if userdata.prev_agent:
    items_copy = self._truncate_chat_ctx(
        userdata.prev_agent.chat_ctx.items, 
        keep_function_call=True,  # Preserve function calls
        keep_last_n_messages=200
    )
    existing_ids = {item.id for item in chat_ctx.items}
    items_copy = [item for item in items_copy if item.id not in existing_ids]
    chat_ctx.items.extend(items_copy)
• Or, set 
exclude_function_call=False
when using `copy`:
Copy code
if isinstance(userdata.prev_agent, Agent):
    truncated_chat_ctx = userdata.prev_agent.chat_ctx.copy(
        exclude_instructions=True, 
        exclude_function_call=False  # Preserve function calls
    ).truncate(max_items=6)
    existing_ids = {item.id for item in chat_ctx.items}
    items_copy = [item for item in truncated_chat_ctx.items if item.id not in existing_ids]
    chat_ctx.items.extend(items_copy)
Manual addition of function calls is not required if you use these built-in context copying mechanisms with the correct parameters. This will ensure that all function calls and their outputs are preserved for new agents, maintaining the necessary context for multi-agent workflows. The complete conversation history is also available in 
session.history
if you need to reference it directly. Sources: complex-agents/personal_shopper/personal_shopper.py | livekit-agents/livekit/agents/voice/agent.py | Agents v0.x migration guide | LiveKit Docs | Workflows | LiveKit Docs
c
I don't use copy nor _truncate_chat_ctx, in the handoff tool I just return the agent with his config that inherits from the previous agent with a simple chat_ctx=chat_ctx
t
Hey there! Let me break down how 
chat_ctx=chat_ctx
works during agent handoff. It's pretty cool - this method keeps all your chat history intact, including those function calls and responses. So no worries about losing important context when switching agents! But, I want to give you a heads up about a few things to keep in mind: • Context Length Management: While keeping the whole chat history sounds great, it can get a bit overwhelming as your conversation grows. Most production systems actually trim things down to keep just the most relevant messages. • Orphaned Function Calls: Without some smart trimming, you might end up with some lonely function calls floating around at the start of your history - not ideal! • Duplicate Prevention: When you're merging contexts, using explicit copying lets you avoid repeating stuff by checking item IDs. Super helpful! • Selective Filtering: The 
copy()
method is your friend here - it lets you get super precise about what context you want to keep. Your current approach totally works, but if you're looking to level up your multi-agent system, I'd recommend giving explicit copying and truncation a try. It'll help you keep things tidy, manageable, and running smoothly as your system grows. Trust me, your future self will thank you! Sources: complex-agents/medical_office_triage/README.md | livekit-agents/livekit/agents/voice/agent.py
c
no my current approach doesn't work because as I said, and it's the reason why I'm here, the function calls, and function calls outputs are not passed to the next agent after handoff in the chat context, only the user and assistant messages are
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
3 Views
Open in Slack
PreviousNext
Powered by