https://livekit.io logo
Join Slack
Powered by
How many agents can i have with the same name to h...
# ask-ai
e
How many agents can i have with the same name to help with server load and do you have any examples of this
t
Thanks for reaching out, Devin. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks for reaching out! LiveKit doesn’t set a limit on how many agents can share the same name, which gives you some flexibility. Just a quick note: when you’re registering an agent, make sure to set the 
agent_name
field in 
WorkerOptions
if you want explicit dispatch. Keep in mind, if you use 
agent_name
, the agent won’t be automatically dispatched to newly created rooms. If you have a particular scenario in mind or need a bit more clarity, please don’t hesitate to ask—I’m here to help you every step of the way! Sources: examples/src/multi_agent.ts | examples/agent_dispatch.py | Dispatching agents | LiveKit Docs | examples/voice_agents/README.md
I'm glad you're exploring how to handle hundreds of simultaneous users chatting with the same agent! A great way to achieve this is by deploying and scaling your agent using LiveKit Cloud. This platform thoughtfully manages load balancing, capacity, and resource allocation for you, so it smoothly supports high concurrency scenarios. Here are some key features that might interest you: • Stateful Load Balancing: Keeps each session on the same server to reduce latency and jitter, ensuring a smoother experience. • Elastic Capacity Management: Automatically scales agent instances to comfortably handle any traffic spikes. • Operational Observability: You can easily monitor performance through the LiveKit Cloud dashboard. • Cost Model: Each active agent session costs $0.01 per minute, covering hosting and data transfer. To get started, you can use the LiveKit Agents framework, which is designed to support multi-user scenarios. For instance, the 
multi-user-transcriber.py
agent creates individual sessions for each participant in a room. Plus, you can test how well it scales using the 
agent-load-test
utility available in the LiveKit CLI. If you'd like, I'm here to help with more details or guide you through setting this up. Just let me know! Sources: Deploy and scale agents on LiveKit Cloud | Going beyond a single-core | examples/other/transcription/multi-user-transcriber.py | README.md
e
I'm thinking of having hundreds of simultaneous users talking to the same agent from what I understood that would require multiple instances of the same agent_name
t
I'd be happy to guide you through configuring multiple instances of the same 
agent_name
in LiveKit. Just follow these simple steps: 1. Set the 
agent_name
in WorkerOptions: Make sure to specify the same 
agent_name
for all instances that you want to include in the same pool. For example:
Copy code
opts = WorkerOptions(
    agent_name="test-agent",
)
1. Run multiple instances: Launch several agent worker instances. Each one will register with LiveKit using the same 
agent_name
. 2. Automatic load balancing: LiveKit will then distribute incoming jobs for that 
agent_name
across all your available instances seamlessly. 3. Explicit dispatch: If you want to dispatch agents to rooms explicitly, you can use the 
AgentDispatchService
API or CLI like this:
Copy code
lkapi = api.LiveKitAPI()
dispatch = await lkapi.agent_dispatch.create_dispatch(
    api.CreateAgentDispatchRequest(
        agent_name="test-agent", 
        room=room_name
    )
)
1. Scaling: Feel free to scale your instances up or down based on your workload. LiveKit takes care of registration and load balancing automatically, so you don’t have to worry. This setup lets you easily scale your agent processing capacity horizontally while keeping a single 
agent_name
for your clients to connect to. If you have any questions or need more help, just let me know—I’m here for you! Sources: Dispatching agents | LiveKit Docs | Agent dispatch | LiveKit Docs | Deployment and scaling | LiveKit Docs
e
this is all I need even with 10 agents all with the same name and different agent id's? lkapi = api.LiveKitAPI() dispatch = await lkapi.agent_dispatch.create_dispatch( api.CreateAgentDispatchRequest( agent_name="test-agent", room=room_name ) )
t
Absolutely, even if you have 10 agents all using the same 
agent_name
but with different agent IDs, this setup will work smoothly. LiveKit is designed to automatically balance the incoming jobs for that 
agent_name
across all the registered instances. Each agent will have its own unique ID, yet they’ll work together to manage the workload for the shared 
agent_name
. Your dispatch code looks spot on for explicitly assigning agents with the 
agent_name
to a specific room. If you’d like, I’m here to help clarify anything further or assist you with the next steps!
e
can you explain this LiveKit takes care of registration and load balancing automatically, so you don’t have to worry.
t
I'm glad you asked about deploying multiple agents with the same 
agent_name
! LiveKit makes this process really smooth by automatically handling their registration and load balancing. Let me walk you through how it works: • Registration: Each time an agent instance starts, it registers itself with LiveKit. When several instances share the same 
agent_name
, they’re grouped together into a pool for that name. • Load Balancing: LiveKit thoughtfully distributes incoming jobs or tasks for that shared 
agent_name
across all the instances in the pool. This way, no single instance gets overwhelmed while others are waiting around. • Scaling: If you add or remove agent instances, LiveKit smoothly adjusts the pool and keeps balancing the load for you—no need to step in manually. This handy automation lets you focus on what matters most—your application logic—while LiveKit takes care of the behind-the-scenes work of scaling and balancing agent workloads. If you have any more questions or need further help, I’m here for you!
e
So if I wanted 10 agents I would run lk create 10 times
t
Absolutely! To deploy 10 agents, you'll just need to run the agent creation process 10 times—either by doing it manually or using an automated script. Each agent will register with LiveKit under the same 
agent_name
, and LiveKit will take care of balancing the load across all 10 instances for you. If you'd like, I’m here to help you automate this process or answer any other questions you might have. Just let me know!
e
How do i efficiently find out the load a single agent can handle till I need to start adding 2, 3, 4, 5 agents with the same name?
t
To help you figure out how much load a single agent can comfortably handle before it's time to scale up, here’s a friendly step-by-step guide: 1. Default Load Management: LiveKit keeps an eye on CPU usage by default, and an agent will pause accepting new jobs once CPU usage hits 70%. If you want, you can tweak this behavior using the 
load_fnc
and 
load_threshold
settings in 
WorkerOptions
. 2. Hardware Recommendations: A good starting point is 4 CPU cores and 8GB of memory per worker. Usually, this setup can handle about 10-25 concurrent jobs, depending on how complex the agent’s tasks are. 3. Load Testing: I recommend using the LiveKit CLI’s 
lk perf agent-load-test
command to simulate multiple concurrent rooms and see how your setup performs under real conditions. Here’s an example:
Copy code
lk perf agent-load-test \
  --rooms 5 \
  --agent-name test-agent \
  --echo-speech-delay 10s \
  --duration 5m
1. Cloud Deployments: If you’re using LiveKit Cloud, it’s great because it automatically scales agent instances based on load, so you won’t need to worry about manually adjusting capacity. By running these load tests and keeping an eye on performance, you’ll get a clear sense of when it’s time to add more agents to keep up with demand. If you’d like, I’m here to help you set up those load tests or configure your agents—just let me know! Sources: Worker options | LiveKit Docs | Deploying to custom environments | LiveKit Docs | pkg/loadtester/agentloadtester.go | Deploy and scale agents on LiveKit Cloud
e
will the agents actually speak in these simulations? Does that need to be taken into account?
t
Thanks for your question! When you run simulations using the 
lk perf agent-load-test
command, the agents aren’t actually speaking or processing real audio. Instead, this tool creates a simulated environment with participants and agents to help measure how your system performs under load. So, it doesn’t include the extra processing work involved in things like noise cancellation, speech-to-text, or other AI tasks. If your agents are handling resource-heavy tasks like real-time audio processing or AI-driven interactions, it’s important to keep that extra load in mind when figuring out your system’s capacity. You might want to try running some custom tests with real workloads to get a clearer picture of what your system can handle. I’m here to help if you’d like some guidance on setting those up!
e
so what if for instance i wanted to load balance with 2 agents per room so they can talk to each other can I do that?
t
Absolutely! LiveKit can definitely handle load balancing and support two agents in the same room so they can chat with each other. To make this happen smoothly, a few specific settings need to be in place: • Enable Agent-to-Agent Communication: You’ll want to configure the agents to detect each other’s speech. This can be done by setting 
listen_to_agents=True
in 
RoomInputOptions
or by including 
AGENT
in 
participant_kinds
. Here’s a quick example:
Copy code
room_io = RoomIO(
    session,
    room=ctx.room,
    input_options=RoomInputOptions(
        listen_to_agents=True  # Allows agents to hear each other
    )
)
• Text-to-Speech (TTS): Make sure both agents have TTS set up so they can produce audible responses. • Permissions: Double-check that both agents have the rights to publish and subscribe to audio tracks within the room. • Load Balancing: LiveKit takes care of distributing agents across worker instances automatically to keep things running efficiently. With these configurations, your two agents will be able to interact seamlessly in the same room—perfect for automated conversations or testing scenarios. If you’d like, I’m here to help you walk through the setup or answer any questions you might have! Sources: Anatomy of an Agent | LiveKit Docs | How to Enable Agent-to-Agent Speech Detection in LiveKit | Agent Testing & Evaluation | livekit/livekit_composite | DeepWiki | An open source stack for real-time multimodal AI | Deployment and scaling | LiveKit Docs
e
How many rooms can a single agent usually be in at the same time?
t
Hey there! Just to help you out, a single agent can join multiple rooms at the same time. How many rooms they can be in really depends on your LiveKit plan: • Build Plan (Free Tier): Allows up to 5 concurrent agent sessions. • Scale Plan or Custom Plan: Offers higher limits depending on your subscription. Keep in mind, each room an agent joins uses one session slot. If you're using a self-hosted setup, the limit depends on your infrastructure's capacity instead of a fixed number. You can easily keep track of and manage these limits by heading over to the LiveKit Cloud dashboard under Settings > Project. If anything’s unclear or you want a hand with this, just let me know—I’m here to help! Sources: Quotas and limits | LiveKit Docs | LiveKit Pricing
e
Ship 2024: Monthly subscription is that my subscription?
t
Thank you for reaching out! Could you please help me understand a bit more about what you mean by "Ship 2024: Monthly subscription"? Are you asking about a particular LiveKit plan, a feature, or maybe something related to billing? Any extra details you can share will really help me assist you better.
e
i want to know how many rooms an agent can be in with this plan
t
Thanks for reaching out! The number of rooms an agent can be in at the same time really depends on the concurrency limits set by your LiveKit plan. If you're using the Build plan (which is the Free tier), the default limit is 5 concurrent agent sessions. For plans with higher limits, like the Scale plan or custom plans, you'll want to check the details specific to your subscription. Regarding "Ship 2024: Monthly subscription," if that’s a custom or specific plan, I’d suggest taking a look at your LiveKit Cloud dashboard under Settings > Project to see what concurrency limits apply to you. And of course, if you’d like, I’m here to help you navigate through it—just let me know!
e
I have the Ship plan it doesnt say. What do the other plans other than free allow?
t
Thanks for reaching out! For the Ship plan, which starts at $50/month, here’s a quick overview of the concurrency limits: • Concurrent agent sessions: 20 • Agent deployments: 2 • Agent session minutes: 5,000 included monthly, with additional minutes at $0.01 per minute Just to give you a sense of the options, the Scale plan (starting at $500/month) offers 50+ concurrent agent sessions, and the Enterprise plan provides unlimited sessions and deployments. If you find that you need higher limits, you can easily upgrade to a higher plan or request specific limit increases through the LiveKit Cloud dashboard. I’m here to help if you have any questions or need a hand with anything! Sources: LiveKit Pricing | Quotas and limits | LiveKit Docs
e
how much does unlimited cost?
t
I'm happy to help you with the cost details! The unlimited agent sessions and deployments are part of our Enterprise plan, which comes with custom pricing designed to fit your unique needs. You can easily get a quote or more information by reaching out to LiveKit's sales team through the LiveKit Cloud dashboard or their official contact channels. If you'd like, I can guide you through the dashboard or assist you in contacting the sales team for a personalized plan. Just let me know—I’m here to help!
4 Views
Open in Slack
PreviousNext
Powered by