https://livekit.io logo
Join Slack
Powered by
How can i pass agent instruction when dispatching ...
# ask-ai
a
How can i pass agent instruction when dispatching agent or update the agent instruction when agent is live in cloud?
✅ 1
t
Thanks for reaching out, Paweł. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking how to (A) pass instructions when dispatching an agent and (B) update instructions while an agent is live. Below are the options you can use. Passing instructions at dispatch • You cannot pass a full system instruction string directly at dispatch time. • Use job metadata to inject dynamic values at dispatch; the agent can read metadata in its entrypoint and apply those values inside its instructions or logic. • Use templated prompts (e.g., 
{{metadata.key}}
) in your agent builder prompts to substitute metadata into the agent’s base instructions. Updating instructions while the agent is live • Use 
update_instructions()
at runtime to replace the agent’s system prompt immediately without restarting the session. The change is applied to the current session and recorded in conversation history as an AgentConfigUpdate. • For single-reply guidance (transient), call `session.generate_reply(instructions="...")`; those instructions affect only that reply and do not replace the agent’s base instructions. If you want, tell me: • Whether you plan to send simple metadata values or full prompt strings at dispatch, and • Which SDK/language you’re using (Python, JS, etc.), and I’ll provide a short code example tailored to your setup. Sources: Agent dispatch | LiveKit Documentation | Job lifecycle | LiveKit Documentation | Agent Builder | LiveKit Documentation | Prompting guide | LiveKit Documentation | External data and RAG | LiveKit Documentation, with interactive elements
a
I'd love to do it via API over HTTP, no interesting in changing the instruction for the existing sessions, just for the new ones. I have agent in Python
i'm thinking about updating the entire instruction/prompt
t
Thanks — you're creating new agent sessions via HTTP and want per-session instructions (Python agent), without changing existing sessions. You can’t send a full system prompt directly at dispatch, but you can pass job metadata over the HTTP API and have the new session’s agent read that metadata or use templated prompts that interpolate metadata into the agent’s instructions. Quick plan: 1. Send job metadata in your dispatch/create-session HTTP request. 2. In your Python agent entrypoint, read 
ctx.job.metadata
(JSON) and either: ◦ Build the effective system prompt from metadata values, or ◦ Substitute metadata into templated instructions defined in the agent builder. Example flow (HTTP + Python) 1) Dispatch / create the job via HTTP (include metadata) • POST /jobs (or your dispatch endpoint) • Body example (JSON):
Copy code
{
  "agent_name": "my-agent",
  "metadata": {
    "role": "support",
    "greeting": "Hello — I'm here to help with billing."
  },
  "other_job_fields": "..."
}
2) Python agent entrypoint reads metadata and applies it to instructions • In your agent handler, parse 
ctx.job.metadata
, then set the session/system prompt or use those values when generating replies. For example:
Copy code
@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    metadata = json.loads(ctx.job.metadata)
    role = metadata.get("role", "assistant")
    greeting = metadata.get("greeting", "Hello!")

    # Option A: Set effective instructions at session start
    await self.update_instructions(f"You are a {role}. {greeting}")

    # Option B: Use metadata per-reply without changing base instructions
    reply = await session.generate_reply(instructions=f"Use a {role} tone. Start with: {greeting}")
    await session.send(reply)
Notes • Use metadata templating in your agent builder prompts (e.g., 
{{metadata.greeting}}
) if you prefer prompts to be managed in the builder and just supply values via metadata. • 
update_instructions()
affects the current session; since you only create new sessions, calling it at session start sets the instruction for that new session only. • 
session.generate_reply(instructions=...)
is transient (per-reply) and won’t persist beyond that reply. If you want, I can: • Provide a concrete HTTP request example for your specific dispatch endpoint, or • Show a complete minimal Python agent file that reads metadata and sets instructions at session start. Which would you prefer?, with interactive elements
Thanks — I’ll help you update the agent’s full instruction/prompt for new sessions via an HTTP API in Python (without affecting existing live sessions). Summary of approach • Keep the current instruction string in a shared configuration (in-memory, database, or file). • Expose an HTTP endpoint that updates that stored instruction. • When creating a new agent/session, read the current instruction and pass it into the Agent constructor so only future sessions use the new prompt. Minimal example (FastAPI + simple global storage)
Copy code
from fastapi import FastAPI, Body
from livekit_agents import Agent, server, JobContext  # adjust imports to your code

app = FastAPI()
current_instructions = "Default instructions for new sessions"

class Assistant(Agent):
    def __init__(self, instructions: str):
        super().__init__(instructions=instructions)

@app.post("/update-instructions")
def update_instructions(payload: dict = Body(...)):
    global current_instructions
    current_instructions = payload["instructions"]
    return {"status": "ok", "instructions_preview": current_instructions[:120]}

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    # Read the latest instructions when creating the agent for this session
    assistant = Assistant(instructions=current_instructions)
    session = assistant.create_session(...)  # your existing session startup
    await session.start(agent=assistant, room=ctx.room)
Notes and options • Persist the instruction in a DB or file if you need it to survive restarts instead of a global variable. • This pattern ensures existing sessions keep their original instructions because each session is created with the instruction value read at creation time. • If you ever want transient per-session overrides, you can also pass metadata or use 
generate_reply(instructions=...)
for single replies (but that’s per-reply, not a global change). Want a version that uses a database for persistence or a complete runnable example tied to your current agent code? Share which web framework and persistence you prefer and I’ll produce it. Sources: Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Build a salary negotiation coach with a LemonSlice avatar | LiveKit | Agent speech and audio | LiveKit Documentation | Gemini Live API plugin | LiveKit Documentation, with interactive elements
a
i'm using livekit cloud to host my agent
there is no db there, right?
t
You're right to ask — LiveKit Cloud doesn't provide a built-in database for storing custom agent instructions. Options to persist and update instructions for new sessions while hosting on LiveKit Cloud: • Short-term (simple): keep instructions in memory (a global variable). Works while the process is running but will be lost on restarts or if the process is rescheduled. • Persistent (recommended): store instructions in an external service you control (examples): ◦ Managed key/value store (Redis, AWS Parameter Store, AWS SSM, Google Secret Manager) ◦ Object store or file in cloud storage (S3/GCS) ◦ A small external DB (RDS, DynamoDB, Firestore, etc.) • Hybrid: write to a file on local disk and periodically back up to remote storage — less reliable than a true external store. Minimal pattern to implement (you said Python + HTTP API): 1. Expose an HTTP endpoint that accepts the new full instruction/prompt. 2. Save the instruction to your chosen store. 3. When a new session starts, read the current instruction from that store and pass it into the Agent constructor so only new sessions use it. If you want, I can: • Provide a FastAPI example that uses Redis (or S3, or a local JSON file) for persistence. • Show how to deploy that example alongside your LiveKit Cloud agent runtime (brief steps). Which persistence option would you prefer for an example (Redis, S3, local file, or something else)? , with interactive elements
7 Views
Open in Slack
PreviousNext
Powered by