I would like to use LiveKit for voice chat, and possibly video, with the option of having agents join and leave dynamically. That is, the lifetime of the agent would not be tied to the lifetime of the LiveKit room. How do I do this?

Thanks for reaching out, Mat. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

My room management is in C# in Unity and my agent is in Python. I'd like code for both sides for dispatching agents dynamically.

Thanks — I’ve put together a concise, working pattern you can use: Unity/C# will dispatch an agent to a specific room, and the Python agent will receive that dispatch and join the room to handle audio/video. I’ll include: • A Unity C# example that calls the AgentDispatch API (HTTP POST) to dispatch an agent to a named room. • A Python agent example that registers with the same agent name and runs as an agent job entrypoint, connects to the dispatched room, and then leaves when finished. Before you start: confirm these details so I tailor the code to your environment • Your LiveKit server base URL and whether you use Twirp or REST endpoint for AgentDispatch calls. • How you authenticate API calls from Unity (API key/secret or a server-signed Bearer token). • The agent name you want to use (both sides must match). If any of those aren’t decided yet I can structure the examples generically and note where to insert your values. Unity (C#) — dispatch an agent to a room • What this does: Sends an authenticated POST to the AgentDispatchService CreateDispatch endpoint to request an agent named

my-unity-agent

join

roomName

. • Replace placeholders with your LiveKit URL and authorization token.

using System.Collections;
using System.Text;
using UnityEngine;
using UnityEngine.Networking;
using System;

[Serializable]
public class DispatchAgentRequest
{
    public string agent_name;
    public string room;
    public string metadata; // optional JSON string
}

public class AgentDispatcher : MonoBehaviour
{
    // Fill these values with your environment specifics
    public string livekitBaseUrl = "<https://your-livekit.example>"; // no trailing slash
    public string apiAuthToken = "REPLACE_WITH_YOUR_BEARER_TOKEN_OR_API_KEY";
    public string agentName = "my-unity-agent";

    public void Dispatch(string roomName, string metadataJson = null)
    {
        StartCoroutine(DispatchAgentCoroutine(agentName, roomName, metadataJson));
    }

    private IEnumerator DispatchAgentCoroutine(string agentName, string roomName, string metadataJson)
    {
        var payload = new DispatchAgentRequest
        {
            agent_name = agentName,
            room = roomName,
            metadata = metadataJson
        };

        string jsonBody = JsonUtility.ToJson(payload);

        // Twirp endpoint path for AgentDispatchService/CreateDispatch — adjust if using REST or a different path
        string url = $"{livekitBaseUrl}/twirp/livekit.AgentDispatchService/CreateDispatch";

        using (UnityWebRequest www = new UnityWebRequest(url, "POST"))
        {
            byte[] bodyRaw = Encoding.UTF8.GetBytes(jsonBody);
            www.uploadHandler = new UploadHandlerRaw(bodyRaw);
            www.downloadHandler = new DownloadHandlerBuffer();
            www.SetRequestHeader("Content-Type", "application/json");

            // Use whatever auth mechanism you have — here we add an Authorization header
            www.SetRequestHeader("Authorization", $"Bearer {apiAuthToken}");

            yield return www.SendWebRequest();

#if UNITY_2020_1_OR_NEWER
            if (www.result != UnityWebRequest.Result.Success)
#else
            if (www.isNetworkError || www.isHttpError)
#endif
            {
                Debug.LogError($"Agent dispatch failed: {www.error}\nResponse: {www.downloadHandler.text}");
            }
            else
            {
                Debug.Log($"Agent '{agentName}' dispatched to room '{roomName}'. Response: {www.downloadHandler.text}");
            }
        }
    }
}

Usage in Unity: • Attach AgentDispatcher to a GameObject. • Call Dispatch(roomName, metadataJson) when you want to send an agent to a room. • metadataJson is optional; you can pass a JSON string with contextual data (user id, call id, routing info). Notes for Unity side • Use a secure server-to-server token for the Authorization header. Do not embed sensitive API secrets inside a shipped Unity client. • If your LiveKit deployment requires a different endpoint (non-Twirp), change the URL path accordingly. • Add retry/backoff and proper error handling for production. Python agent — receive dispatch and join the room • What this does: The agent registers with name

my-unity-agent

, handles the dispatched job, connects to the room, optionally uses audio/video (auto-subscribe), and then shuts down cleanly when done.

import asyncio
import logging
from livekit import agents

logging.basicConfig(level=<http://logging.INFO|logging.INFO>)
logger = logging.getLogger("my-unity-agent")

# Use the decorator with the same agent_name used in Unity
@agents.rtc_session(agent_name="my-unity-agent")
async def agent_entrypoint(job: agents.JobContext):
    # Log job metadata if provided
    if job.job.metadata:
        <http://logger.info|logger.info>(f"Dispatch metadata: {job.job.metadata}")

    # Connect this agent session to the room (auto_subscribe enables audio/video subscriptions)
    await job.connect(auto_subscribe=agents.AutoSubscribe.AUDIO_VIDEO)

    # Wait for participants or handle events — wait_for_participant returns first non-agent participant
    try:
        participant = await job.wait_for_participant(timeout=30)  # timeout in seconds
        <http://logger.info|logger.info>(f"Connected with participant: {participant.identity}")
    except asyncio.TimeoutError:
        <http://logger.info|logger.info>("No participant joined within timeout. Proceeding with agent logic or shutting down.")
        # Optionally shut down early if no participants
        await job.shutdown()
        return

    # Example: produce a simple reply or run your conversation logic
    await job.generate_reply(instructions="Welcome the user and offer help with their request.")

    # Keep the agent alive to handle audio events, messages or run longer tasks
    # This pattern just waits for a condition or a fixed time, adjust to your needs
    try:
        # Example: wait until job is asked to stop or for N seconds
        await asyncio.wait_for(job.wait_for_shutdown(), timeout=300)  # wait up to 5 minutes
    except asyncio.TimeoutError:
        <http://logger.info|logger.info>("Agent job timeout reached, shutting down.")
        await job.shutdown()

if __name__ == "__main__":
    # Run the agent worker process
    worker = agents.Worker(job_request_handler=agent_entrypoint)
    agents.run_app(worker)

Important Python notes • The decorator

@agents.rtc_session(agent_name="my-unity-agent")

means the agent is explicitly named and will accept explicit dispatches for that name. •

job.connect(auto_subscribe=agents.AutoSubscribe.AUDIO_VIDEO)

ensures the agent subscribes to audio/video if you want the agent to hear and (optionally) publish audio/video. • Use

job.wait_for_participant()

to wait for a regular participant to arrive in the room. Agents are treated as non-essential participants, so room lifetime is unaffected by agent disconnects. • Implement robust error handling & reconnect logic in production. Agent shutdown and room lifecycle • You can programmatically shut down the agent job from inside the Python code with `await job.shutdown()`; doing so will remove the agent from the room without ending the room for other participants. • Rooms are created automatically when the first participant with a valid token joins; you do not need to call CreateRoom() manually. Next steps I can help with • Provide a production-ready Unity server flow that generates a short-lived server token (so you don’t ship API keys in a client). • Add an example of passing structured metadata (e.g., JSON with user_id, skill routing) and show how the Python agent can use it for routing or personalization. • Show code for the Python agent to publish audio (TTS) back into the room if you want the agent to speak. Which follow-up would you like first: a secure token flow for Unity (server-side signing) or a metadata/routing example for agent matching? Sources: Server options | LiveKit Documentation

In the Unity side of your example I think I've got the wrong string for livekitBaseUrl. If my project were called, for example, "zebra-12345678" what would livekitBaseUrl be?

Thanks — good question. You’re right to check the URL string. For a LiveKit Cloud project named "zebra-12345678": • Use the WebSocket URL for client room connections in Unity: ◦ wss://zebra-12345678.livekit.cloud • If you need an HTTP(S) base URL for server-to-server API calls (e.g., AgentDispatch API), use: ◦ https://zebra-12345678.livekit.cloud Quick notes: • In Unity when connecting a Room, pass the WebSocket URL and a participant token:

await room.Connect("<wss://zebra-12345678.livekit.cloud>", "participant-token");

• For dispatching agents from your server (HTTP POST to AgentDispatchService), use the HTTPS base and the correct endpoint path (adjust if your deployment uses Twirp vs REST): ◦ Example Twirp path: https://zebra-12345678.livekit.cloud/twirp/livekit.AgentDispatchService/CreateDispatch • Never embed API secrets in the shipped Unity client—generate short-lived participant tokens server-side and send those to the client. If you’d like, I can: • Show the exact Unity connection call with token creation flow (server-side C# or Node/Python example), or • Give the full AgentDispatch POST example using the correct HTTPS base URL for your project. Which would help most? Sources: Configure ZScaler proxy for LiveKit connections | LiveKit | examples/survey | livekit/livekit_composite | DeepWiki

I need Unity code now to get the right token for this call.

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

Should the participant token used for CreateDispatch be the same a the participant token used to create a room?