I want to update system message/LLM instructions after a function call. I want to get the chat context updated immediately so that LLM picks the new context in its response generation post function call. Suggest a approach

Thanks for reaching out, Deepak. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

using python SDK. Issue I am facing is that chat context is not updated in the turn following function call. It gets updated in the next turn

Thanks — this is clear and I can help. You’re using the Python SDK and the chat context update only takes effect one turn later instead of immediately after the function call. I’ll give targeted troubleshooting steps, a concrete Python example pattern you can copy, and a few diagnostic questions so I can narrow down the root cause if these don’t fix it. Quick checklist (most common causes) • Ensure you await the context update call (async). If you don’t await, the next LLM turn may run before the update completes. • Make sure the next LLM invocation fetches the agent’s current ChatContext (no stale local cache). • If you recreate the ChatContext, include prior messages in the correct order and put the new system message first so it takes precedence. • Confirm there’s no intermediate cache or queued request (for example, a queued LLM call started before update finished). Suggested pattern (async Python) 1. Update the chat context inside the function/tool. 2. Await the update so it completes before returning. 3. Trigger the next LLM turn only after update completes. Example pattern you can adapt:

@function_tool()
async def set_system_instructions(self, context: RunContext):
    # 1) Compose the new system instruction
    new_system = "You are now in expert mode. Provide detailed technical advice."

    # 2) Build an updated ChatContext with system message first
    updated_ctx = ChatContext()
    updated_ctx.add_message(role="system", content=new_system)

    # 3) Re-add prior conversation in order (optional but recommended)
    for msg in self.chat_ctx.get_messages():
        updated_ctx.add_message(role=msg.role, content=msg.content)

    # 4) Persist the update and await it so it finishes before this function returns
    await self.update_chat_ctx(updated_ctx)

    # 5) Return something the caller can inspect if needed
    return "ok"

How to verify it applied immediately • After the function returns, explicitly call the agent/LLM for the next reply only after awaiting the function result. • Log or inspect the agent’s current chat context immediately after

await self.update_chat_ctx(...)

to confirm the system message is present. • If you call the LLM from a different coroutine/task, ensure that coroutine starts only after the update completes. Diagnostic questions (so I can help further if the pattern above doesn’t fix it) • Are you calling the next LLM turn from the same process/thread as the update, or from a separate worker/process? • Do you use any local caching of

chat_ctx

or keep a copy of it in memory that might be stale? • Are you sure

await self.update_chat_ctx(...)

is being executed (no exceptions swallowed)? • Which exact Python SDK method are you using to send the next user prompt / trigger the next LLM response? If you share a short snippet showing: • the function/tool that performs the update, and • how you trigger the next LLM call right after the function returns, I’ll point out exactly where timing or ordering needs to change and give a targeted fix.

what is difference between

update_chat_ctx

and

update_instructions

? I am using

update_instructions.

Thanks — I’m glad you asked. I’ll explain the difference and which to use so your instruction/context changes take effect immediately after a function call. Short answer • Use update_instructions when you want to change the agent’s system-level instructions (the “role” or behavior). This directly updates the system prompt the LLM uses. • Use update_chat_ctx when you want to change the conversation history (messages) the LLM sees next turn. Why that matters for immediate effect •

update_instructions

changes the system prompt itself, so the LLM will use the new instructions on the very next generation once that call completes. •

update_chat_ctx

modifies the stored chat history. It should also take effect immediately if you await it and the next LLM call reads the latest chat context — but if you’re seeing a one-turn delay, it’s often because the context update wasn’t awaited, a stale cached copy of the context was used, or the next LLM call started before the update finished. Which to use for your goal • If your goal is to change the LLM’s behavior or role (for example: “now respond as an expert”): use

update_instructions

. • If your goal is to inject new conversational content or persist data into the conversation history (for example: saving answers collected during a function call so the model references them later): use

update_chat_ctx

. Practical tips to ensure immediate effect • Await the update call (e.g.,

await agent.update_instructions(...)

or

await agent.update_chat_ctx(...)

) so the update completes before returning from the function. • After awaiting, trigger the next LLM request only once the update call has finished. • If you use

update_chat_ctx

, ensure you update the in-memory/chat object you use to build the next prompt so no stale cached version is sent. • If you need the model to change behavior right away, prefer

update_instructions

because it directly replaces the system prompt. If you want, share the short snippet where you call

update_instructions

and then trigger the next LLM turn — I’ll point out any ordering or awaiting issues and give a tiny edit to guarantee the update takes effect immediately. Sources: Building an Automated IVR Menu Caller | LiveKit Documentation | Interrupt User | LiveKit Documentation | Agents and handoffs | LiveKit Documentation

I am already using

await update_instructions

but the chat context is not updated in the very next time, though it gets updated afterwards

Thanks — I hear you: you call

await update_instructions(...)

but the very next model response still uses the old context and only reflects the new instructions one turn later. That’s frustrating. I’ll explain exactly why that happens and give two concrete, minimal fixes you can apply in Python so the change is effective immediately. Why this happens (short) •

update_instructions

changes the agent’s system-level prompt for future turns, but it does not automatically re-inject that new prompt into an already-prepared/queued LLM generation. If the next generation was started (or the session context was prepared) before the instruction update finished, that generation will still use the old instructions. Quick checklist (confirm these) • You are awaiting the call:

await self.update_instructions(...)

. • You are not starting the next LLM call from a different coroutine/task that began before the await finished. • You are not relying on a cached copy of

chat_ctx

when building the next prompt. Two concrete ways to guarantee immediate effect Option A — Rebuild and update the chat context (recommended for immediate, deterministic effect) • Add the new system instruction as the first message in a rebuilt ChatContext, persist it with

await self.update_chat_ctx(...)

, then call the LLM. • This ensures the next generation sees the new system message because you control exactly what is sent. Example pattern:

@function_tool()
async def my_tool(self, context: RunContext):
    new_instr = "You are now in expert mode. Provide detailed, technical answers."

    # Rebuild ChatContext with the new system message first
    new_ctx = ChatContext()
    new_ctx.add_message(role="system", content=new_instr)

    # copy prior non-system messages in order
    for msg in self.chat_ctx.get_messages():
        if msg.role != "system":
            new_ctx.add_message(role=msg.role, content=msg.content)

    # persist and await so it completes before next generation
    await self.update_chat_ctx(new_ctx)

    # now generate the next reply — it will use the updated context
    await context.session.generate_reply("Please respond under new instructions.")

Option B — Use update_instructions but ensure the next generation starts only after the update completes • If you want to use

update_instructions

, make sure you do not trigger or allow any LLM call to start until the update has fully completed and the session/agent has refreshed. • That often means: await

update_instructions()

, then return from the tool and let the next user/agent turn occur (or explicitly trigger the follow-up only after the await completes). Pattern:

@function_tool()
async def my_tool(self, context: RunContext):
    await self.update_instructions("You are now concise and formal.")
    # Do NOT start a generation here in the same event loop tick
    return "instructions-updated"  # let the next cycle perform the generation

Extra diagnostic pointers (if it still lags) • Confirm the next LLM call is created after the await, not started concurrently in another task. • Log the agent/session chat context immediately after the await to confirm the new instruction is present. • If you use worker processes or multiple agent instances, ensure you update the same agent/session that will do the next generation. • If you do rebuild chat context, put the new system message at the top so it takes precedence. If you want, share a minimal snippet showing: • where you call

await update_instructions(...)

, and • how/where you immediately trigger the next LLM call I’ll point to the exact line to change so the new instructions take effect in the very next response. Sources: Building multi-agent architectures with LiveKit agents | LiveKit | Agents and handoffs | LiveKit Documentation | Building an Automated IVR Menu Caller | LiveKit Documentation

yes, point me to exact line to change.

Thanks — to point exactly to the line you need to change I need the small snippet where you: 1. Call

await update_instructions(...)

, and 2. Immediately trigger the next LLM generation (for example

context.session.generate_reply(...)

,

agent.generate_reply(...)

,

send_user_message(...)

, or similar). Please paste those 8–20 lines (the function/tool and the immediate follow-up call). I’ll mark the exact line to change and give the minimal edit you should apply so the new instructions take effect in the very next response.