Thanks — this is a great, specific question about runtime updates to agent context. I’ll answer the part about updating the system prompt between turns and whether that can cause unexpected behavior. Short answer • LiveKit supports swapping instructions (system prompt) at runtime and records those changes in the conversation history as an AgentConfigUpdate. • What’s not documented is how LiveKit handles updates that occur concurrently with an in-progress generation. There is no documented guarantee that updates are atomic or that mid-generation updates won’t affect that same in-flight response. • To avoid any possible race conditions, update instructions only after the prior agent turn has finished generating. What we know (behavior you can rely on) • You can change an agent’s instructions at runtime using APIs like

update_instructions()

or similar, and those changes are tracked in the conversation history so the model can see configuration changes between turns. • When instructions or tools change, an

AgentConfigUpdate

entry is appended to the conversation, listing updated instructions and tools added/removed. What is uncertain / not documented • There is no explicit documentation about: ◦ Whether

update_instructions()

is atomic with respect to an ongoing generation. ◦ Whether a mid-generation update could partially influence the current response or be ignored until the next turn. ◦ Any internal locking, queuing, or synchronization that prevents race conditions during LLM inference. Recommended safe approach 1. Wait for the first agent turn to complete (for example, after

generate_reply()

returns or in

on_user_turn_completed

) before swapping the system prompt. 2. If you need lower startup latency, consider: ◦ Pre-warming or caching the agent with the most commonly used prompt ahead of time (if that fits your flow). ◦ Performing prompt updates only at deterministic handoff points rather than during generation. 1. If you must perform updates concurrently, build application-level coordination (locks or state flags) so you don’t call

update_instructions()

while a generation is in progress. If you’d like, I can help with: • Specific code patterns for waiting on turn completion in your environment. • An example coordination pattern (locks/flags) to avoid concurrent updates. • Brainstorming startup-latency strategies that don’t require mid-generation instruction swaps. Which of those would be most helpful for you? Sources: Build a salary negotiation coach with a LemonSlice avatar | LiveKit | Build a salary negotiation coach with a LemonSlice avatar | LiveKit | Building an Automated IVR Menu Caller | LiveKit Documentation | Function tool definition | LiveKit Documentation | Agents and handoffs | LiveKit Documentation, with interactive elements