how many agents can one worker handle, for production purposes?

Thanks for reaching out, Arthur. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

So for example, a 1000 concurrent users, would require many workers

Hey! For handling 1,000 concurrent users, you'll want to have multiple workers running. Typically, each worker supports between 10 to 25 agents, but the exact number really depends on your machine’s specs and setup. Just to give you an idea: • If one worker manages 25 agents, you'd be looking at around 40 workers. • If one worker manages 10 agents, then about 100 workers would do the trick. It’s a good idea to keep an eye on your resource usage and add more workers as you go, so everything runs smoothly without a hitch.

If I were to do explicit dispatch, how would it be spread across different workers, so the load is balanced?

Hey there! If you want to balance the load across workers using explicit dispatch in LiveKit, here’s a simple way to do it: 1. Enable Explicit Dispatch: Just set the

agent_name

in

WorkerOptions

. This turns off automatic dispatch and gives you manual control over which agents get assigned where. 2. Create Explicit Dispatches: Use the

AgentDispatchService

API to assign agents to specific rooms. Here’s a quick example:

import asyncio
from livekit import api

room_name = "my-room"
agent_name = "test-agent"

async def create_explicit_dispatch():
    lkapi = api.LiveKitAPI()
    dispatch = await lkapi.agent_dispatch.create_dispatch(
        api.CreateAgentDispatchRequest(
            agent_name=agent_name, 
            room=room_name, 
            metadata='{"user_id": "12345"}'
        )
    )
    print("Dispatch created:", dispatch)

1. Keep an Eye on Worker Load: Set up

load_fnc

and

load_threshold

in

WorkerOptions

so a worker knows when to stop taking on new jobs based on how busy it is. 2. Use LiveKit’s Built-in Job Distribution: The system automatically assigns jobs in a round-robin style and smartly reroutes them if a worker is unavailable or overloaded. By keeping track of worker loads and using explicit dispatch, you’ll have a smooth and efficient way to balance work across your team. If you have any questions or need a hand, just let me know!