waht are the different options to update the llm i...
# ask-ai
m
waht are the different options to update the llm instructions like chatcontext?
t
Thanks for reaching out, Rajadurai. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Here are the different options to update LLM instructions like chat context in LiveKit: • Explicit Update: Use the
agent.update_chat_ctx()
method to directly modify the chat context. This works for both realtime and non-realtime sessions. • Agent Handoff: Transfer control to a new agent, which creates a new chat context for the new agent. • Update Instructions: Use the
update_instructions()
method to inject new instructions at the beginning of the chat context or update existing ones. • On User Turn Completed Node: For STT-LLM-TTS pipelines, use the
on_user_turn_completed
node to perform a RAG lookup and dynamically inject additional context into the chat context. Each method is suited for different use cases depending on your requirements. Sources: Agents v0.x migration guide | LiveKit Docs | External data and RAG | LiveKit Docs | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/real...
m
I dont want to tottaly change the prompt, I need to add some instruction on it
t
If you want to add instructions to an existing chat context without replacing the prompt, you can use the
update_instructions
method. This handy method either updates the current instruction message or adds new instructions if none are there yet. Here’s a quick rundown: • Update Existing Instructions: Find the instruction message in the chat context and tweak its content. • Add Instructions if Missing: If there’s no instruction message, simply insert new instructions right at the start of the chat context. For realtime sessions, make sure to use the
update_instructions
method from the
AgentActivity
class so everything stays synced properly. Once you’ve made your changes, don’t forget to update the chat context with
update_chat_ctx
. Feel free to ask if you’d like me to share some detailed code examples or explain anything further—I’m here to help! Sources: livekit-agents/livekit/agents/voice/agent_activity.py
m
waht are the diffrent roles in Chatmessage when to use it
t
Here’s a quick rundown of the different roles in
ChatMessage
and when you’d typically use each one: • System Role: This is for system-generated messages, like when setting things up or giving initialization instructions. Think of it as laying the groundwork for the conversation. • User Role: This one’s for messages sent by real people. For example, when you’re typing a message in a chat room. • Assistant Role: This role is for AI-generated replies, like those from chatbots or language models. It’s the AI’s way of chatting back with you. • Tool Role: Used when messages come from external tools or function calls. For instance, if a tool is sharing data or results based on what you asked. These roles really help keep things clear and organized by showing where each message is coming from. If you want me to walk you through more examples or details, just let me know—I’m here to help! Sources: agents/src/llm/chat_context.ts