how to pass user metadata like jwt token detils from fronted and get metadata in agent entrypoint

Thanks for reaching out, arpan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

/**

* LiveKit Service * Handles token generation and room connection for voice agents */ const LIVEKIT_URL = import.meta.env.VITE_LIVEKIT_URL || "wss://test-wazew3xi.livekit.cloud"; const LIVEKIT_API_KEY = import.meta.env.VITE_LIVEKIT_API_KEY || "APIwVaLzakZvFa3"; const LIVEKIT_API_SECRET = import.meta.env.VITE_LIVEKIT_API_SECRET || "Z6apyLgqccQVqfRNc07OZcIMOkUp7kcZhSQdbQtgNWE"; export interface LiveKitConfig { serverUrl: string; token: string; roomName: string; } /** * Generate a random room identifier */ function generateRoomId(): string { return `holionix-${Date.now()}-${Math.random().toString(36).substring(7)}`; } /** * Generate a LiveKit access token * Note: In production, this should be done server-side for security * For demo purposes, we're using the Access Token directly with credentials */ export async function getLiveKitToken( participantName: string = "User", roomName?: string ): Promise<LiveKitConfig> { const room = roomName || generateRoomId(); // For now, we'll use a simple JWT generation approach // In production, call your backend API endpoint const identity = `user-${Date.now()}`; // Import AccessToken from livekit-server-sdk // This is a temporary solution - move to backend in production try { const { AccessToken } = await import('livekit-server-sdk'); const at = new AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET, { identity: identity, ttl: '10m', }); at.addGrant({ roomJoin: true, room: room, canPublish: true, canSubscribe: true, canPublishData: true, }); at.name = participantName; const token = await at.toJwt(); return { serverUrl: LIVEKIT_URL, token, roomName: room, }; } catch (error) { console.error('Error generating token:', error); throw new Error('Failed to generate LiveKit token. Please ensure livekit-server-sdk is installed.'); } } /** * Validate LiveKit configuration */ export function validateLiveKitConfig(): boolean { if (!LIVEKIT_URL || !LIVEKIT_API_KEY || !LIVEKIT_API_SECRET) { console.error('LiveKit configuration is missing. Please check your .env file.'); return false; } return true; }

Thanks — this is helpful. You’ve shared your LiveKit token-generation code and want to pass user metadata (like JWT token details) from the frontend through your backend/token and then access it from the agent entrypoint. I’ll show a secure, practical approach that fits your Node/TypeScript flow and your existing code. Short plan • Don’t generate tokens in the browser in production — generate them on a backend endpoint. • Have the frontend send the user metadata (either safe JWT claims or a reference like user_id) to the backend call that issues the LiveKit token. • Embed the metadata as JSON in the dispatch or in the token’s room agent metadata field. • In the agent entrypoint, parse

ctx.job.metadata

(string) as JSON to read the passed values. What to change in your flow (concrete steps) 1) Move token generation to a backend endpoint (recommended) • Create an endpoint (e.g., POST /api/connection-details) that receives from the frontend the participantName, roomName (optional), and the metadata you want to pass (e.g., user_id, roles, or verified JWT claims). • Generate the AccessToken on the server using your existing logic (you already have the AccessToken usage) — do NOT expose LIVEKIT_API_SECRET to the browser. 2) Decide what to pass as metadata • Prefer passing verified, non-sensitive claims (sub, email, roles). Do NOT place secrets or raw long-lived tokens in metadata. • If you must pass a sensitive value, instead pass a reference (user_id) and have the agent call your backend to fetch the sensitive data during the job (backend can authenticate/authorize that call). 3) Embed metadata in the token / dispatch • Two common approaches: ◦ A: Include metadata in the token room configuration for the agent (RoomConfiguration.agents[].metadata) ◦ B: Create an agent dispatch and set the dispatch metadata field • Both store metadata as a string — serialize JSON. Example server-side pattern (TypeScript/Node pseudocode that fits your current code) 1. Backend endpoint receives frontend request:

// POST /api/connection-details
// body: { participantName, roomName?, metadata } 
// metadata example: { user_id: '123', email: '<mailto:alice@example.com|alice@example.com>' }

import { AccessToken } from 'livekit-server-sdk';

<http://app.post|app.post>('/api/connection-details', async (req, res) => {
  const { participantName = 'User', roomName, metadata } = req.body;
  const room = roomName || generateRoomId();

  // ensure metadata is safe: whitelist fields or only pass verified claims
  const safeMetadata = JSON.stringify({
    user_id: metadata?.user_id,
    email: metadata?.email,
    roles: metadata?.roles
  });

  const at = new AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET, {
    identity: `user-${Date.now()}`,
    ttl: '10m',
  });

  at.addGrant({
    roomJoin: true,
    room,
    canPublish: true,
    canSubscribe: true,
    canPublishData: true,
  });

  at.name = participantName;

  // Option: include agent metadata inside token room_config
  // If your SDK supports RoomConfiguration on the token, set it here.
  // Otherwise use agent dispatch API to attach metadata to the job.

  const token = await at.toJwt();

  res.json({ serverUrl: LIVEKIT_URL, token, roomName: room, metadata: safeMetadata });
});

If you use an agent dispatch API instead of token room_config: • Call the dispatch endpoint from your backend, passing

metadata: safeMetadata

in the dispatch body. The agent job will receive that metadata. 4) Read metadata in the agent entrypoint • In your agent code, parse

ctx.job.metadata

(it’s a string). Example: Node/TypeScript agent entry:

export default defineAgent({
  entry: async (ctx: JobContext) => {
    let metadata = {};
    if (ctx.job?.metadata) {
      try {
        metadata = JSON.parse(ctx.job.metadata);
      } catch (err) {
        console.warn('Invalid job metadata JSON', err);
      }
    }
    const userId = metadata.user_id;
    const email = metadata.email;
    // Use these to customize behavior or fetch more info from your backend
  }
});

Python agent entry (if applicable):

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    metadata = {}
    if ctx.job.metadata:
        metadata = json.loads(ctx.job.metadata)
    user_id = metadata.get("user_id")

Best practices & security reminders • Do token creation on the backend — never ship LIVEKIT_API_SECRET to clients. • Only include non-sensitive, verified claims in metadata. If you need secrets, use a server-side lookup by passing a safe reference. • Keep metadata small and focused. Use JSON string serialization. • Validate and whitelist fields in metadata on the backend before embedding. If you want, I can: • Convert the backend pseudocode into a complete Express + TypeScript endpoint using your exact environment variables. • Show an example of dispatching an agent with metadata (if you’re using LiveKit’s agent dispatch API). • Show how to securely validate a JWT on the backend and extract safe claims to pass as metadata — tell me whether you want to pass the whole JWT or only claims.

my api in fastapi in diffirent server and voice agent diffirent server

Thanks — this is helpful context. You're running your FastAPI backend on one server and your voice agent on a different server. I’ll explain a secure, practical pattern you can follow and then ask a couple quick clarifying questions so I can give code that fits your setup. Short summary of recommended flow • Frontend → call your FastAPI backend with user info or verified JWT claims. • FastAPI (server A) → validate JWT, extract safe claims (or generate a short-lived reference), then either: ◦ create an agent dispatch with metadata pointing to the voice agent job (preferred if you can call the dispatch API), or ◦ generate a LiveKit token that includes the metadata (if your token flow supports embedding agent metadata). • Voice agent (server B) → receives the job with

ctx.job.metadata

(metadata is whatever you put in the dispatch or token). Parse JSON and, if necessary, call your FastAPI backend to fetch additional user data securely using the safe reference. Why this works for distributed servers • Metadata travels with the agent job (not via the browser), so the voice agent on a separate server can read it directly from the JobContext. • Your FastAPI server keeps secrets (LIVEKIT_API_SECRET) and validates/authenticates user tokens; the agent server never needs the LiveKit secret. • If sensitive data is required, send a reference (user_id or short-lived lookup token) in metadata and have the agent fetch the sensitive parts from FastAPI over a secure internal API. Concrete guidance and options Option A — FastAPI creates the agent dispatch with metadata (recommended) • Flow: a. Frontend POST -> FastAPI with JWT (or extracted claims). b. FastAPI validates JWT, whitelists claims, builds safe metadata JSON (e.g., { user_id, email, roles } or { user_ref: "ref-abc123" }). c. FastAPI calls the agent dispatch API (LiveKit or your agent system) and includes metadata in the dispatch request. d. Voice agent later starts and reads

ctx.job.metadata

. • Benefits: Direct mapping of user -> agent job, no secrets shared to clients. Option B — FastAPI generates token with metadata (if your token supports room_config agent metadata) • Flow: a. Frontend -> FastAPI with JWT. b. FastAPI validates claims and embeds safe metadata into the token’s room_config or returns both token + metadata reference. c. Agent job reads

ctx.job.metadata

when the dispatch is created by whoever uses that token. • Use only if your token flow supports embedding agent metadata. Option C — Minimal metadata + server-to-server lookup (most secure for sensitive data) • Pass only a small reference like

user_id

or a short-lived lookup token in metadata. • Voice agent calls FastAPI internal endpoint (authenticated using mTLS, API key, or short-lived token) to fetch full profile or validate permissions. Security best practices • Keep LIVEKIT_API_SECRET on FastAPI server only. • Validate and whitelist JWT claims on FastAPI; do not blindly pass full JWTs to agent jobs. • Do not put raw secrets or long-lived tokens in metadata. • Protect server-to-server calls between agent and FastAPI (use mutual TLS, signed short-lived tokens, or scoped API keys). • Keep metadata small and JSON-serialized. Example patterns (high-level) 1) FastAPI endpoint to receive frontend request and create dispatch (pseudo) • Validate JWT and build safeMetadata = json.dumps({ user_id, roles }) • Call dispatch API: { agent_name: "voice-agent", room: "room-id", metadata: safeMetadata } 2) Agent entrypoint (server B) • Parse metadata: const metadata = ctx.job.metadata ? JSON.parse(ctx.job.metadata) : {}; const userId = metadata.user_id; • If more info needed: fetch(

<https://api.yourdomain.com/internal/user/${userId}>

, { headers: { Authorization:

Bearer ${INTERNAL_API_KEY}

} }) Quick clarifying questions 1. Does your FastAPI server currently have permission to call the agent dispatch API (or the LiveKit agent dispatch endpoint), or do you need to route through another service? 2. Do you prefer the agent to receive full verified claims, or just a reference (user_id) and fetch details from FastAPI? 3. Which authentication method can the agent use to call FastAPI for lookups? (API key, mTLS, short-lived signed token, etc.) Tell me your answers and I’ll give a concrete FastAPI + agent code example (complete with request/response shapes and a secure lookup pattern) that fits your preferred option.

can you provide a exampel repo aor code to understand more

Thanks — I’ll put together a minimal end-to-end example you can run locally (or adapt) that shows FastAPI on one server and a voice agent on another, with metadata flowing from the frontend → FastAPI → LiveKit agent. I’ll keep it concise and focused so you can copy/paste. What I’ll provide: • FastAPI server: validates a (demo) JWT claim, creates a LiveKit AccessToken and attaches agent metadata via an Agent Dispatch API call (recommended), or includes metadata in a RoomConfiguration if you prefer token-based dispatch. • Agent server: an example agent entrypoint (Node or Python) that parses

ctx.job.metadata

. • Frontend snippet: how to call FastAPI and connect to LiveKit. • Notes on secure production practices. Before I produce code: one quick question so I tailor the repo exactly to your setup • Do you want the agent example in Python or Node (TypeScript)? Reply with "python" or "node" and I’ll produce the full example files. If you want the Python variant, I’ll provide: • fastapi_server/main.py • agent_server/agent.py • frontend example (minimal fetch) • instructions to run both servers and test metadata flow If you want Node, I’ll provide: • fastapi_server/main.py (FastAPI still in Python) • agent_server/index.ts (Node agent) • frontend example and run instructions Tell me which agent language you prefer and I’ll generate the full example code and run steps. Sources: Agent dispatch | LiveKit Documentation | Job lifecycle | LiveKit Documentation | Python SDKs | livekit/livekit_composite | DeepWiki | Survey Calling Agent | LiveKit Documentation

Python variant please

Thanks — I’ll give you a minimal, runnable Python example that demonstrates the full flow: • Frontend sends user info to FastAPI. • FastAPI validates (demo) JWT claims, creates a LiveKit AccessToken, and dispatches an agent with JSON job metadata. • Voice agent (separate Python process) receives the job and reads metadata from

ctx.job.metadata

. Repository layout (minimal) • fastapi_server/ ◦ main.py ◦ requirements.txt ◦ .env.example • agent_server/ ◦ agent.py ◦ requirements.txt ◦ .env.example • frontend/ ◦ index.html ◦ script.js • README.md (run steps below) Important notes before running • This example is for local/dev testing. Do NOT expose LIVEKIT_API_SECRET to clients in production. • Replace placeholders in .env files with real credentials (LIVEKIT API key/secret and LiveKit base URL). • The agent dispatch method used here calls the LiveKit Agent Dispatch API (server-to-server). The agent server reads metadata from JobContext. 1) fastapi_server/main.py

# fastapi_server/main.py
import os
import json
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from dotenv import load_dotenv
import httpx

load_dotenv()

LIVEKIT_URL = os.getenv("LIVEKIT_URL")  # e.g. "https://<your-project>.livekit.cloud"
LIVEKIT_API_KEY = os.getenv("LIVEKIT_API_KEY")
LIVEKIT_API_SECRET = os.getenv("LIVEKIT_API_SECRET")
AGENT_DISPATCH_URL = os.getenv("AGENT_DISPATCH_URL")  # e.g. "https://<your-project>.livekit.cloud/v1/agent/dispatch"

if not (LIVEKIT_URL and LIVEKIT_API_KEY and LIVEKIT_API_SECRET and AGENT_DISPATCH_URL):
    raise RuntimeError("Set LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET, AGENT_DISPATCH_URL in .env")

app = FastAPI()
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

# Simple demo model: frontend sends a demo JWT (string) and user info.
class TokenRequest(BaseModel):
    demo_jwt: str
    user_id: str
    user_name: str
    room_name: str = None
    agent_name: str = "demo-agent"

def generate_room(room_name: str | None):
    if room_name:
        return room_name
    import time, random
    return f"room-{int(time.time())}-{random.randint(1000,9999)}"

@app.post("/create-session")
async def create_session(req: TokenRequest):
    # DEMO: validate the demo_jwt in a trivial way.
    if not req.demo_jwt or "demo" not in req.demo_jwt:
        raise HTTPException(status_code=401, detail="Invalid demo_jwt")

    room = generate_room(req.room_name)

    # Build safe metadata (whitelist fields)
    safe_metadata = {
        "user_id": req.user_id,
        "user_name": req.user_name,
        # Don't send secrets. Include only safe claims or references.
    }
    metadata_str = json.dumps(safe_metadata)

    # Create an agent dispatch by calling LiveKit Agent Dispatch API (server-to-server)
    # The API call below is a minimal example. Confirm endpoint and auth method for your LiveKit setup.
    dispatch_payload = {
        "agent_name": req.agent_name,
        "room": room,
        "metadata": metadata_str,
        # additional fields as needed by your LiveKit deployment
    }

    # Basic auth with API key/secret for LiveKit admin endpoints (depends on your LiveKit deployment)
    async with httpx.AsyncClient() as client:
        # For many deployments, LiveKit admin endpoints require the API key in headers and a JWT. If your LiveKit expects different auth,
        # adjust accordingly. This example uses an endpoint protected by API Key/Secret via basic auth (demo).
        try:
            resp = await <http://client.post|client.post>(
                AGENT_DISPATCH_URL,
                json=dispatch_payload,
                auth=(LIVEKIT_API_KEY, LIVEKIT_API_SECRET),
                timeout=10.0
            )
        except Exception as e:
            raise HTTPException(status_code=500, detail=f"Dispatch call failed: {e}")

    if resp.status_code not in (200, 201):
        raise HTTPException(status_code=500, detail=f"Dispatch API responded {resp.status_code}: {resp.text}")

    # Return room info to frontend. Frontend will use its own path to get a LiveKit token to join,
    # or you can additionally generate a LiveKit access token server-side and return it here.
    return {
        "room": room,
        "agent_dispatched": True,
        "job_metadata": safe_metadata,
        "dispatch_response": resp.json()
    }

fastapi_server/requirements.txt

fastapi
uvicorn
python-dotenv
httpx
pydantic

fastapi_server/.env.example

LIVEKIT_URL=<https://your-project.livekit.cloud>
LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret
# LiveKit agent dispatch endpoint - adjust for your deployment:
AGENT_DISPATCH_URL=<https://your-project.livekit.cloud/v1/agent/dispatch>

Notes: • The example uses a simple HTTP POST to LiveKit dispatch API with basic auth. Your LiveKit deployment might require a signed JWT admin token or different auth; adapt the request accordingly. • Alternatively, instead of calling dispatch API, you can generate a token containing RoomConfiguration with agent metadata (if your SDK supports) and return that token to the frontend. Dispatch via API is a clear server-to-server way to attach metadata. 2) agent_server/agent.py

# agent_server/agent.py
import os
import json
import asyncio
from dotenv import load_dotenv
load_dotenv()

# This is a minimal demo agent using LiveKit Agents Python SDK utilities (pseudo).
# Replace with the actual agent SDK imports and runner you use in production.
# The important part: ctx.job.metadata contains the JSON string you passed.

# PSEUDO SDK placeholders. Replace with concrete imports from your agent runtime.
# For this example, we'll show a simple loop that pretends to be the agent job runner.

class DemoJobContext:
    def __init__(self, job):
        self.job = job

async def run_agent_entrypoint(ctx: DemoJobContext):
    print("Agent started; parsing job metadata...")
    metadata_raw = ctx.job.get("metadata")
    if metadata_raw:
        try:
            metadata = json.loads(metadata_raw)
        except Exception as e:
            print("Failed to parse metadata JSON:", e)
            metadata = {}
    else:
        metadata = {}

    user_id = metadata.get("user_id")
    user_name = metadata.get("user_name", "Guest")

    print(f"Agent job for user_id={user_id}, user_name={user_name}")
    # Example behavior: agent greets user and waits (demo)
    print(f"Agent: Hello {user_name}! I received your metadata.")
    await asyncio.sleep(2)
    print("Agent: finishing job.")

# Demo runner: in a real setup the agent runtime would invoke entrypoint with real JobContext
if __name__ == "__main__":
    # Demo: simulate receiving a job payload from LiveKit agent system.
    # For testing, you can run: python agent.py '{"metadata":"{\"user_id\":\"123\",\"user_name\":\"Alice\"}"}'
    import sys
    if len(sys.argv) > 1:
        job_json = json.loads(sys.argv[1])
    else:
        # fallback demo metadata
        job_json = {"metadata": json.dumps({"user_id": "demo-1", "user_name": "Demo User"})}

    ctx = DemoJobContext(job_json)
    asyncio.run(run_agent_entrypoint(ctx))

agent_server/requirements.txt

python-dotenv

agent_server/.env.example

# Agent-specific env values if needed
AGENT_NAME=demo-agent

3) frontend/index.html + script.js • A minimal static page that posts to FastAPI to create a session/dispatch. frontend/index.html

<!doctype html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>LiveKit metadata demo</title>
  </head>
  <body>
    <h1>Frontend Demo</h1>
    <button id="start">Create session & dispatch agent</button>
    <pre id="out"></pre>
    <script src="script.js"></script>
  </body>
</html>

frontend/script.js

document.getElementById('start').onclick = async () => {
  const demoJwt = "demo-token-abc"; // demo token for validation on FastAPI
  const payload = {
    demo_jwt: demoJwt,
    user_id: "user-123",
    user_name: "Alice",
    room_name: "" // optional
  };

  const res = await fetch("<http://localhost:8000/create-session>", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  const data = await res.json();
  document.getElementById('out').innerText = JSON.stringify(data, null, 2);
};

4) Running locally (quick steps) • Start FastAPI

aur bhai kaisa hai @polite-dog-12152

Main theek hoon, shukriya — aap kaise ho? Aapko LiveKit / FastAPI agent integration par aur madad chahiye kya? Agar koi specific file, code snippet, ya next steps chahiye (jaise full repo zip ya run instructions), bataiye — main wohi provide kar dunga.

@millions-gold-86808 achai hu upp kasa hai

Main theek hoon, shukriya — aap kaise ho, Arpan? Agar aap chahte hain toh main abhi aap ke liye poora runnable repo scaffold kar ke de sakta hoon (FastAPI backend + Python agent + minimal frontend) aur run instructions bhi doonga. Ya phir agar aapko sirf kisi particular file (FastAPI endpoint, agent entrypoint, or dispatch call) ka complete code chahiye to woh bhi bhej sakta hoon. Batayein: • Full repo chahiye ya specific files? • Kya aap agent ko real LiveKit project pe test karna chahte hain ya locally simulate karna?

Badiya, where do you work at ?

I’m a virtual assistant working with the LiveKit support team here to help you with integration and troubleshooting. I don’t have a physical office — I’m here to help you any time you need guidance on LiveKit, FastAPI, or agent integrations. If you want, I can: • Scaffold the full runnable repo (FastAPI backend + Python agent + minimal frontend) and provide step-by-step run instructions. • Or send specific complete files (FastAPI endpoint, agent entrypoint, and a sample dispatch call). Which would you prefer?

now open to work

Great — since you said "now open to work" and asked earlier for the Python variant, I’ve prepared a minimal, runnable Python example you can copy locally. It demonstrates the full flow: • Frontend → FastAPI (validates demo JWT, creates dispatch with metadata) • FastAPI → calls LiveKit agent-dispatch API with JSON job metadata • Voice agent (separate Python process) → reads metadata from the job entrypoint I'll give you the full repository layout, each file's content, and step-by-step run instructions. This example is intended for local/dev testing and uses a simplified dispatch call; adapt auth details for your LiveKit deployment. Repository layout • repo-root/ ◦ fastapi_server/ ▪︎ main.py ▪︎ requirements.txt ▪︎ .env.example ◦ agent_server/ ▪︎ agent.py ▪︎ requirements.txt ▪︎ .env.example ◦ frontend/ ▪︎ index.html ▪︎ script.js ◦ README.md Files 1) fastapi_server/main.py

# fastapi_server/main.py
import os
import json
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from dotenv import load_dotenv
import httpx

load_dotenv()

LIVEKIT_URL = os.getenv("LIVEKIT_URL")  # e.g. "https://<your-project>.livekit.cloud"
LIVEKIT_API_KEY = os.getenv("LIVEKIT_API_KEY")
LIVEKIT_API_SECRET = os.getenv("LIVEKIT_API_SECRET")
AGENT_DISPATCH_URL = os.getenv("AGENT_DISPATCH_URL")  # e.g. "https://<your-project>.livekit.cloud/v1/agent/dispatch"

if not (LIVEKIT_URL and LIVEKIT_API_KEY and LIVEKIT_API_SECRET and AGENT_DISPATCH_URL):
    raise RuntimeError("Set LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET, AGENT_DISPATCH_URL in .env")

app = FastAPI()
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

class TokenRequest(BaseModel):
    demo_jwt: str
    user_id: str
    user_name: str
    room_name: str = None
    agent_name: str = "demo-agent"

def generate_room(room_name: str | None):
    if room_name:
        return room_name
    import time, random
    return f"room-{int(time.time())}-{random.randint(1000,9999)}"

@app.post("/create-session")
async def create_session(req: TokenRequest):
    # DEMO validation of demo_jwt (replace with real JWT validation in prod)
    if not req.demo_jwt or "demo" not in req.demo_jwt:
        raise HTTPException(status_code=401, detail="Invalid demo_jwt")

    room = generate_room(req.room_name)

    # Whitelist fields to include in metadata
    safe_metadata = {
        "user_id": req.user_id,
        "user_name": req.user_name,
    }
    metadata_str = json.dumps(safe_metadata)

    dispatch_payload = {
        "agent_name": req.agent_name,
        "room": room,
        "metadata": metadata_str,
    }

    # Server-to-server call to LiveKit Agent Dispatch API
    async with httpx.AsyncClient() as client:
        try:
            resp = await <http://client.post|client.post>(
                AGENT_DISPATCH_URL,
                json=dispatch_payload,
                auth=(LIVEKIT_API_KEY, LIVEKIT_API_SECRET),
                timeout=10.0
            )
        except Exception as e:
            raise HTTPException(status_code=500, detail=f"Dispatch call failed: {e}")

    if resp.status_code not in (200, 201):
        raise HTTPException(status_code=500, detail=f"Dispatch API responded {resp.status_code}: {resp.text}")

    return {
        "room": room,
        "agent_dispatched": True,
        "job_metadata": safe_metadata,
        "dispatch_response": resp.json()
    }

fastapi_server/requirements.txt

fastapi
uvicorn
python-dotenv
httpx
pydantic

fastapi_server/.env.example

LIVEKIT_URL=<https://your-project.livekit.cloud>
LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret
AGENT_DISPATCH_URL=<https://your-project.livekit.cloud/v1/agent/dispatch>

2) agent_server/agent.py

# agent_server/agent.py
import os
import json
import asyncio
from dotenv import load_dotenv
load_dotenv()

# This is a minimal demo showing how an agent would parse ctx.job.metadata.
# Replace DemoJobContext and runner with your actual agent runtime integration.

class DemoJobContext:
    def __init__(self, job):
        self.job = job

async def run_agent_entrypoint(ctx: DemoJobContext):
    print("Agent started; parsing job metadata...")
    metadata_raw = ctx.job.get("metadata")
    if metadata_raw:
        try:
            metadata = json.loads(metadata_raw)
        except Exception as e:
            print("Failed to parse metadata JSON:", e)
            metadata = {}
    else:
        metadata = {}

    user_id = metadata.get("user_id")
    user_name = metadata.get("user_name", "Guest")

    print(f"Agent job for user_id={user_id}, user_name={user_name}")
    print(f"Agent: Hello {user_name}! I received your metadata.")
    await asyncio.sleep(2)
    print("Agent: finishing job.")

if __name__ == "__main__":
    import sys
    if len(sys.argv) > 1:
        job_json = json.loads(sys.argv[1])
    else:
        job_json = {"metadata": json.dumps({"user_id": "demo-1", "user_name": "Demo User"})}

    ctx = DemoJobContext(job_json)
    asyncio.run(run_agent_entrypoint(ctx))

agent_server/requirements.txt

python-dotenv

agent_server/.env.example

AGENT_NAME=demo-agent

3) frontend/index.html

<!doctype html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>LiveKit metadata demo</title>
  </head>
  <body>
    <h1>Frontend Demo</h1>
    <button id="start">Create session & dispatch agent</button>
    <pre id="out"></pre>
    <script src="script.js"></script>
  </body>
</html>

frontend/script.js

document.getElementById('start').onclick = async () => {
  const demoJwt = "demo-token-abc"; // demo token for validation on FastAPI
  const payload = {
    demo_jwt: demoJwt,
    user_id: "user-123",
    user_name: "Alice",
    room_name: "" // optional, let server generate
  };

  const res = await fetch("<http://localhost:8000/create-session>", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  const data = await res.json();
  document.getElementById('out').innerText = JSON.stringify(data, null, 2);
};

4) README.md (run steps) • Create two terminal windows. • Setup FastAPI server a. cd fastapi_server b. cp .env.example .env and fill LIVEKIT_* values c. python -m venv .venv && source .venv/bin/activate d. pip install -r requirements.txt e. uvicorn main:app --reload --port 8000 • Setup Agent (simulate agent job) a. cd agent_server b. cp .env.example .env c. python -m venv .venv && source .venv/bin/activate d. pip install -r requirements.txt e. To simulate receiving a job from LiveKit, run: python agent.py '{"metadata":"{\"user_id\":\"user-123\",\"user_name\":\"Alice\"}"}' 1. In production, your agent runtime will receive the real JobContext and call an entrypoint similar to run_agent_entrypoint. • Test from frontend a. Open frontend/index.html in a browser (serve statically or open file). b. Click "Create session & dispatch agent". c. The page will POST to http://localhost:8000/create-session and show the dispatch response. Security & production notes (important) • Do NOT expose LIVEKIT_API_SECRET to the frontend. Keep it only on server-side. • Validate JWT properly on FastAPI (this demo uses a simple substring check). • Only include safe, whitelisted claims in job metadata. For sensitive data, pass a short-lived reference (user_id) and have agent call your FastAPI internal endpoint (authenticate via API key, mTLS, or signed short-lived token) to fetch details. • Your LiveKit deployment might require different admin auth for dispatch endpoints (signed admin JWT instead of basic auth). Confirm and adapt the AGENT_DISPATCH_URL and auth method. What I can do next for you • Scaffold a ZIP or GitHub repo with the exact files above so you can clone quickly. • Replace the simplified dispatch auth with the exact method your LiveKit deployment needs (if you tell me how your LiveKit admin endpoints are authenticated). • Convert the agent example to

Ohh so you're an undergrad i assume ?

Haha — I don’t have a school or degree — I’m here as part of the LiveKit support assistant to help you with integrations and code. If you want, we can keep it casual, but I’ll focus on helping you get the FastAPI + LiveKit agent example working. Do you want me to: • Scaffold a GitHub-ready repo with the exact files I posted (zipped or repo layout), or • Send the remaining run instructions and the exact files for the token-generation path (so frontend can also get a LiveKit access token), or • Replace the simplified dispatch auth in the FastAPI example with the exact admin auth your LiveKit deployment uses (tell me whether your LiveKit admin endpoints expect basic auth, a signed admin JWT, or something else)? Tell me which option you prefer and I’ll prepare it right away.