big-school-62939
03/11/2026, 5:55 PMtall-airline-16756
03/11/2026, 5:56 PMtall-airline-16756
03/11/2026, 5:56 PM// 1. Fetch token from your server (HTTP GET/POST)
string token = await FetchTokenFromServer(roomName, participantName);
// 2. Create a new Room and connect
var room = new Room();
// Optional: configure room options here
// Connect using the token (this call is asynchronous in the SDK)
await room.Connect("wss://<your-livekit-url>", token);
// Now you are joined to the room. Publish tracks or subscribe to remote tracks as needed.
If you want, I can give a concrete snippet matching the exact Unity LiveKit SDK methods you’re using (for example, Room.Connect signature, track creation, or handling remote participants). Tell me which Unity LiveKit package/version you have and whether you’re targeting WebGL or native Unity.
Agent-side guidance (server)
• Implement the agent as a backend service using the Agents SDK (Python or Node) or a server LiveKit client.
• Generate a token for the agent on your server (same approach as for Unity), and start the agent process to connect to the room using that token.
• The agent will operate independently and can publish/subscribes just like any client.
Next steps I can help with
• Provide a concrete Unity C# code snippet targeting the exact Unity LiveKit SDK version you’re using.
• Show how to implement a simple agent connection using the Python or Node Agents SDK (sample code).
• Help with your server token generation endpoint (what fields to include in the token request for room, identity, and permissions).
Which would you like next — a full Unity C# example with actual SDK method names, or a sample agent implementation (Python/Node) to run on your server? Also tell me the Unity LiveKit SDK version you’re using.
Sources: Server startup modes | LiveKit Documentation | Agent dispatch | LiveKit Documentation | Agents (Python) | livekit/livekit_composite | DeepWiki | Real-time audio and video in the Metaverse | Simple Call Answering Agent | LiveKit Documentationbig-school-62939
03/11/2026, 6:01 PMtall-airline-16756
03/11/2026, 6:01 PMbig-school-62939
03/11/2026, 6:29 PMtall-airline-16756
03/11/2026, 6:30 PM// Example: node server using LiveKit server SDK
// npm install @livekit/server-sdk (example package name; adjust if different)
const {AgentDispatchServiceClient, CreateDispatchRequest} = require('@livekit/server-sdk');
const grpc = require('@grpc/grpc-js'); // if SDK uses grpc under the hood
// Configure
const LIVEKIT_URL = 'your-livekit-server:443'; // host with port (no scheme)
const API_KEY = process.env.LIVEKIT_API_KEY;
const API_SECRET = process.env.LIVEKIT_API_SECRET;
// Create client (SDK may provide helper; using raw gRPC client pattern shown)
const metadata = new grpc.Metadata();
metadata.set('authorization', `Basic ${Buffer.from(`${API_KEY}:${API_SECRET}`).toString('base64')}`);
// If the SDK exposes a client wrapper, use that. Example generically:
const client = new AgentDispatchServiceClient(LIVEKIT_URL, grpc.credentials.createSsl());
async function createDispatch(agentName, roomName, metadataFields = {}) {
return new Promise((resolve, reject) => {
const req = new CreateDispatchRequest();
req.setRoom(roomName);
req.setAgentName(agentName);
// Attach metadata as JSON string or structured fields per API
if (Object.keys(metadataFields).length) {
req.setMetadata(JSON.stringify(metadataFields));
}
client.createDispatch(req, metadata, (err, resp) => {
if (err) return reject(err);
resolve(resp);
});
});
}
// Example usage:
createDispatch('my-agent', 'my-room', {initiatedBy: 'unity-client'})
.then(res => console.log('Dispatch created:', res))
.catch(err => console.error('Dispatch error:', err));
Option B — Call the LiveKit server REST/gRPC API directly using fetch (REST) or gRPC (if REST is available) (If the server exposes a REST endpoint for agent dispatch; if not, use the Node SDK or gRPC as above.)
// Example using REST POST (if LiveKit exposes HTTP endpoint for dispatch).
// Replace with the correct LiveKit dispatch endpoint path for your deployment.
const fetch = require('node-fetch');
async function createDispatchREST(agentName, roomName, metadataFields = {}) {
const url = `<https://your-livekit-host/api/dispatch`;> // adjust path to real endpoint
const body = {
room: roomName,
agent_name: agentName,
metadata: metadataFields
};
const resp = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
// Basic auth using API key:secret — keep secret on server
'Authorization': 'Basic ' + Buffer.from(`${process.env.LIVEKIT_API_KEY}:${process.env.LIVEKIT_API_SECRET}`).toString('base64'),
},
body: JSON.stringify(body),
});
if (!resp.ok) {
const text = await resp.text();
throw new Error(`Dispatch failed: ${resp.status} ${text}`);
}
return resp.json();
}
// Example usage:
createDispatchREST('my-agent', 'my-room', {initiatedBy: 'unity-client'})
.then(res => console.log('Dispatch REST created:', res))
.catch(err => console.error(err));
What I need from you to tailor the snippet
• Are you using the official LiveKit Node server SDK? If so, tell me the package name/version and I’ll provide exact import/usage.
• If not using the SDK, do you prefer a gRPC or REST example? If REST, confirm the correct LiveKit dispatch endpoint on your deployment.
• Any specific fields you want sent with the dispatch (e.g., participant identity, call metadata, routing hints)?
Next steps I can help with
• Provide the exact Node.js code using your LiveKit SDK/package version.
• Show how to expose a secure server endpoint your Unity client will call to request the dispatch (including example request/response).
• Demonstrate handling the CreateDispatch response (agent session id, status) and returning a safe result to the Unity client.
Which would you like me to produce next?big-school-62939
03/12/2026, 7:06 PMtall-airline-16756
03/12/2026, 7:06 PMnpm install @livekit/server-sdk express
2) Server code (Express + LiveKit Node server SDK)
• This example:
◦ Creates an AgentDispatchService client using the SDK.
◦ Exposes POST /dispatch that accepts JSON { room: string, agent_name: string, metadata?: object }.
◦ Calls CreateDispatch with metadata (encoded as JSON string).
◦ Returns a safe response to the client.
// server.js
const express = require('express');
const bodyParser = require('body-parser');
const { AgentDispatchServiceClient } = require('@livekit/server-sdk');
const grpc = require('@grpc/grpc-js');
const app = express();
app.use(bodyParser.json());
// Configure from env
const LIVEKIT_HOST = process.env.LIVEKIT_HOST || 'your-livekit-host:443'; // host[:port]
const LIVEKIT_API_KEY = process.env.LIVEKIT_API_KEY;
const LIVEKIT_API_SECRET = process.env.LIVEKIT_API_SECRET;
if (!LIVEKIT_API_KEY || !LIVEKIT_API_SECRET) {
console.error('Set LIVEKIT_API_KEY and LIVEKIT_API_SECRET in environment');
process.exit(1);
}
// Build basic auth metadata for gRPC calls
function authMetadata() {
const meta = new grpc.Metadata();
const token = Buffer.from(`${LIVEKIT_API_KEY}:${LIVEKIT_API_SECRET}`).toString('base64');
meta.set('authorization', `Basic ${token}`);
return meta;
}
// Create gRPC client for AgentDispatchService
const client = new AgentDispatchServiceClient(
LIVEKIT_HOST,
grpc.credentials.createSsl()
);
// POST /dispatch
// Body: { room: string, agent_name: string, metadata?: object }
<http://app.post|app.post>('/dispatch', async (req, res) => {
try {
const { room, agent_name, metadata } = req.body;
if (!room || !agent_name) {
return res.status(400).json({ error: 'room and agent_name are required' });
}
// Build the CreateDispatchRequest using the SDK's request shape.
// The exact request class helpers vary by SDK version; if the SDK exposes
// a builder, use it. Here we construct the raw request object expected by gRPC.
const createDispatchReq = {
room: room,
agent_name: agent_name,
// AgentDispatchRequest metadata field expects a string or Struct depending on proto;
// encode metadata as JSON string in the `metadata` field so the agent can parse it.
metadata: metadata ? JSON.stringify(metadata) : undefined,
};
// Call CreateDispatch
const metadataHeaders = authMetadata();
client.createDispatch(createDispatchReq, metadataHeaders, (err, response) => {
if (err) {
console.error('CreateDispatch error:', err);
return res.status(500).json({ error: err.message || 'dispatch_error' });
}
// Response contains dispatch details (agent session id, status, etc).
// Return relevant safe fields to the Unity client.
return res.json({
success: true,
dispatch: response, // you may filter fields here before returning
});
});
} catch (e) {
console.error('Dispatch endpoint error:', e);
res.status(500).json({ error: 'internal_server_error' });
}
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Dispatch server listening on ${PORT}`);
});
3) How metadata is handled
• This example puts metadata into the metadata field as a JSON string. Agents receiving the dispatch can parse that JSON.
• If your agent/LiveKit deployment expects structured metadata differently (e.g., proto Struct), adjust encoding accordingly. The key point: include the data in the CreateDispatch request so the agent gets context (participant id, reason, labels, etc).
4) Security considerations
• Authenticate/authorize Unity callers to your /dispatch endpoint (e.g., session cookie, JWT, API token).
• Validate incoming requests to prevent misuse of agent dispatching.
• Log dispatch outcomes and errors for observability.
5) Return values and follow-up
• The createDispatch response includes dispatch/session identifiers and status. Persist any IDs you need for tracking.
• You can extend the endpoint to return only minimal info to the client (e.g., { success: true, dispatch_id }) to avoid leaking internals.
If you’d like, I can:
• Convert the example to use a Promise-based client wrapper (async/await) for cleaner control flow.
• Show an example Unity C# call to this /dispatch endpoint (the safe client call).
• Tailor the request/response fields returned to your Unity client needs (e.g., return a status polling URL or dispatch id).
Which one would you like next?big-school-62939
03/12/2026, 7:11 PMtall-airline-16756
03/12/2026, 7:11 PMusing System;
using System.Collections;
using System.Text;
using UnityEngine;
using UnityEngine.Networking;
[Serializable]
public class DispatchRequest {
public string room;
public string agent_name;
public object metadata; // can be any serializable object (dictionary, anon object)
}
[Serializable]
public class DispatchResponse {
public bool success;
public string error;
// If your server returns dispatch info, map fields here.
// Example: public string dispatch_id;
// Or include a generic object for the dispatch payload:
public object dispatch;
}
public class AgentDispatchClient : MonoBehaviour {
// Set this to your server endpoint
[SerializeField] private string dispatchEndpoint = "<https://your-server.example.com/dispatch>";
// Example call from Unity (coroutine)
public void RequestAgentDispatch(string room, string agentName, object metadata = null, string bearerToken = null) {
var req = new DispatchRequest {
room = room,
agent_name = agentName,
metadata = metadata
};
StartCoroutine(PostDispatchRequestCoroutine(req, bearerToken));
}
private IEnumerator PostDispatchRequestCoroutine(DispatchRequest requestPayload, string bearerToken) {
string json = JsonUtility.ToJson(requestPayload);
using (UnityWebRequest www = new UnityWebRequest(dispatchEndpoint, "POST")) {
byte[] bodyRaw = Encoding.UTF8.GetBytes(json);
www.uploadHandler = new UploadHandlerRaw(bodyRaw);
www.downloadHandler = new DownloadHandlerBuffer();
www.SetRequestHeader("Content-Type", "application/json");
// If you use bearer token auth from Unity to your server, set it here.
if (!string.IsNullOrEmpty(bearerToken)) {
www.SetRequestHeader("Authorization", $"Bearer {bearerToken}");
}
yield return www.SendWebRequest();
#if UNITY_2020_1_OR_NEWER
bool isNetworkError = www.result == UnityWebRequest.Result.ConnectionError;
bool isHttpError = www.result == UnityWebRequest.Result.ProtocolError;
#else
bool isNetworkError = www.isNetworkError;
bool isHttpError = www.isHttpError;
#endif
if (isNetworkError || isHttpError) {
Debug.LogError($"Dispatch request failed: {www.error} (status: {www.responseCode})");
// You can surface this to UI or retry logic
yield break;
}
string respText = www.downloadHandler.text;
Debug.Log("Dispatch response: " + respText);
// Parse response (use JsonUtility or a JSON library like Newtonsoft if response is complex)
try {
DispatchResponse resp = JsonUtility.FromJson<DispatchResponse>(respText);
if (resp == null) {
Debug.LogWarning("Dispatch response could not be parsed as DispatchResponse.");
// Optionally use a JSON library here for dynamic parsing
} else if (resp.success) {
Debug.Log("Agent dispatch requested successfully.");
// Handle dispatch info in resp.dispatch if needed
} else {
Debug.LogWarning($"Dispatch failed: {resp.error}");
}
} catch (Exception ex) {
Debug.LogError("Failed to parse dispatch response: " + ex.Message);
}
}
}
}
Usage example
• From your UI or code call:
◦ dispatchClient.RequestAgentDispatch("my-room", "my-agent", new { initiatedBy = "unity-client", participantId = "user123" }, bearerToken);
Notes on metadata
• The example sends metadata as an object. Unity's JsonUtility has limits (it doesn't serialize dictionaries). For flexible metadata (dictionaries, nested objects), consider using Newtonsoft.Json (Json.NET for Unity) and replace JsonUtility.ToJson / FromJson with JsonConvert.SerializeObject / DeserializeObject.
• Ensure the server decodes the metadata the same way you encoded it (the Node example encodes metadata to JSON string).
Error handling & security
• Authenticate Unity calls to your server (JWT, session cookies, etc.) so only authorized clients can request agent dispatch.
• Implement rate limiting and validation on the server to avoid abuse.
• Log the dispatch response on the server; Unity should receive only minimal safe info.
If you’d like, I can:
• Provide a version using Newtonsoft.Json for flexible metadata/dictionaries.
• Show a Unity UI example to collect room/agent name and call the request.
• Provide an expected JSON schema for the server response so you can map fields into DispatchResponse.
Which would you like next?