there are many types of objects in chat context. Example: message, agent_config_update, function_call, function_call_output, etc. So does livekit filter any types or send some specific types to llm when sending a request to llm for response

Thanks for reaching out, Kushagra. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

suppose, i keep exclude_config_update as true. then will it exclude the config update objects from chat context also. or is it just to filter out these objects when sending the request to llm

Good question — you're asking whether

exclude_config_update=True

actually removes config-update items from the ChatContext or only prevents them from being sent to the LLM. Short answer: It only filters them out when copying/serializing the context (for example, the copy you send to the LLM). It does not delete or remove

agent_config_update

items from the original `ChatContext.items`; those records remain in the full conversation history and are auditable. If you’d like, I can show a short Python or TypeScript snippet that demonstrates using

copy(exclude_config_update=True)

and how the original context still contains the config updates. Which language do you prefer? Sources: Chat context | LiveKit Documentation | Function tool definition | LiveKit Documentation, with interactive elements

can you give me an example of the payload it passes to llm. Suppose there is a system prompt with type message, then a agent_config_update object, then a few message type objects. The agent also has tools with parameters and descriptions. the

exclude_config_update

is true. then what will be the payload it sends to llm for response

Thanks — I’m responding to your request for a concrete example payload sent to the LLM when

exclude_config_update=true

. Below is an example JSON-style payload that LiveKit would send to the LLM in that scenario. It shows the system prompt and subsequent messages included, the agent config update omitted, and the agent tools provided (name, description, parameters). This matches LiveKit behavior:

agent_config_update

items are filtered out when copying/serializing the context but remain in the original ChatContext.

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant that summarizes user requests."
    },
    {
      "role": "user",
      "content": "Can you summarize the following conversation and list action items?"
    },
    {
      "role": "assistant",
      "content": "Sure — please paste the conversation you want summarized."
    },
    {
      "role": "user",
      "content": "Here is the conversation: ..."
    }
  ],
  "tools": [
    {
      "name": "get_user_profile",
      "description": "Fetches a user's profile by ID (returns name, email, preferences).",
      "parameters": {
        "type": "object",
        "properties": {
          "user_id": {
            "type": "string",
            "description": "ID of the user to fetch"
          }
        },
        "required": [
          "user_id"
        ]
      }
    },
    {
      "name": "create_task",
      "description": "Creates a task in the task manager with title, assignee, and due date.",
      "parameters": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "assignee_id": {
            "type": "string"
          },
          "due_date": {
            "type": "string",
            "format": "date"
          }
        },
        "required": [
          "title",
          "assignee_id"
        ]
      }
    }
  ],
  "metadata": {
    "context_items_sent": [
      "message",
      "message",
      "message",
      "message",
      "tools"
    ],
    "excluded_item_types": [
      "agent_config_update"
    ]
  }
}

Notes: • The

agent_config_update

object that existed in the original ChatContext is not present in

messages

because

exclude_config_update=true

was applied when copying/serializing the context for the LLM. • All

message

items (system/user/assistant) are included unless other exclude flags are set. • Tool definitions (name, description, parameters) are included in the payload so the LLM can produce function calls. • The exact wire format may vary depending on the LLM adapter (OpenAI-style

messages

+

functions

, or a different LLM API), but the content and filtering behavior above is what LiveKit produces when

exclude_config_update

is true. If you want the same example formatted for a specific LLM API (OpenAI-style

messages

+

functions

), or a TypeScript/Python snippet that builds this copy with

exclude_config_update: true

, tell me which and I’ll provide it. Sources: Chat context | LiveKit Documentation | Function tool definition | LiveKit Documentation, with interactive elements

and what if exclude config update was false

You're asking what the LLM payload looks like when

exclude_config_update

is false (so config-update items are included). Below is a concise example JSON-style payload showing the system prompt, the

agent_config_update

item included, subsequent messages, and tool definitions.

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant that summarizes user requests."
    },
    {
      "type": "agent_config_update",
      "update": {
        "instructions": "Prefer concise bullet-point summaries.",
        "tools_added": [
          "create_task"
        ],
        "tools_removed": []
      }
    },
    {
      "role": "user",
      "content": "Can you summarize the following conversation and list action items?"
    },
    {
      "role": "assistant",
      "content": "Sure — please paste the conversation you want summarized."
    },
    {
      "role": "user",
      "content": "Here is the conversation: ..."
    }
  ],
  "tools": [
    {
      "name": "get_user_profile",
      "description": "Fetches a user's profile by ID (returns name, email, preferences).",
      "parameters": {
        "type": "object",
        "properties": {
          "user_id": {
            "type": "string",
            "description": "ID of the user to fetch"
          }
        },
        "required": [
          "user_id"
        ]
      }
    },
    {
      "name": "create_task",
      "description": "Creates a task in the task manager with title, assignee, and due date.",
      "parameters": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "assignee_id": {
            "type": "string"
          },
          "due_date": {
            "type": "string",
            "format": "date"
          }
        },
        "required": [
          "title",
          "assignee_id"
        ]
      }
    }
  ],
  "metadata": {
    "context_items_sent": [
      "message",
      "agent_config_update",
      "message",
      "message",
      "message",
      "tools"
    ],
    "excluded_item_types": []
  }
}

Notes: • With

exclude_config_update=false

,

agent_config_update

entries are included in the copied/serialized context sent to the LLM. • The exact wire-format may vary by LLM adapter (e.g., OpenAI

messages

+

functions

), but the config-update content will be present in the payload unless you explicitly exclude it. Sources: Chat context | LiveKit Documentation, with interactive elements

where does livekit build the payload to request llm

You're asking where LiveKit builds the payload that it sends to the LLM. Short answer: LiveKit constructs the LLM request payload inside the LLM implementation when

LLM.chat()

is called with a

ChatContext

. Concretely: • The

ChatContext

is serialized and mapped to the provider format by the LLM adapter (e.g.,

inference.LLM

builds OpenAI-style `{ model, messages, ... }`;

langchain.LLMAdapter

maps to LangChain message types). • This happens during the AgentSession/LLM lifecycle when a conversation turn triggers the LLM call. • You can customize interception or modification by overriding the

llm_node

or implementing a custom

LLM

node to control how the payload is built and sent. If you want, I can point to the exact file/method in the SDK for your language (Python or TypeScript) or provide a short code example showing where to hook in. Which language do you use? Sources: Large language models (LLM) overview | LiveKit Documentation | Bring your LangChain Agents to LiveKit | LiveKit, with interactive elements

where is the code to add agent config update type, is written

You're asking where in the code the

agent_config_update

item is created/added. Short answer:

AgentConfigUpdate

items are produced when an agent's tools or instructions are changed at runtime (for example,

agent.update_tools(...)

or

agent.update_instructions(...)

in Python). The SDK appends an

AgentConfigUpdate

record to the agent's

chat_ctx.items

list at that point. Key details: • This automatic creation/append behavior is implemented in the Python agent APIs (the feature is currently Python-only). • The

AgentConfigUpdate

contains fields like

instructions

,

tools_added

, and

tools_removed

. • These items are stored in the original

ChatContext.items

(auditable history) and are included in LLM payloads unless you copy/truncate the context with

exclude_config_update=True

. If you want, tell me: • which language (Python or TypeScript) and SDK version you’re using, and • whether you want the exact file/method reference or a short code snippet showing

update_tools

/

update_instructions

producing the

AgentConfigUpdate

., with interactive elements

does livekit also change the temperature for llm

Short answer: LiveKit does not change temperature automatically — it uses whatever

temperature

you configure on the LLM (or the provider default if you don't set one). What you can do: • Set temperature when creating the LLM in your AgentSession. • LiveKit passes that value through in the constructed request to the provider. Examples: Python

from livekit.agents import AgentSession
from livekit.plugins import openai

session = AgentSession(
    llm=openai.LLM(
        model="gpt-4.1",
        temperature=0.5
    ),
)

TypeScript

import { voice } from "@livekit/agents";
import * as openai from "@livekit/agents-plugin-openai";

const session = new voice.AgentSession({
  llm: new openai.LLM({
    model: "gpt-4.1",
    temperature: 0.5
  })
});

Notes: • If you omit

temperature

, the provider default applies (varies by provider). • Per-provider allowed ranges differ (e.g., OpenAI 0.0–2.0). Sources: https://docs.livekit.io/agents/models/llm/openai.md| | Cerebras LLM plugin guide | LiveKit Documentation | DeepSeek LLM | LiveKit Documentation | Groq LLM plugin guide | LiveKit Documentation | Google Gemini LLM | LiveKit Documentation, with interactive elements