Hi, I am planning to create a dashboard for voice agents, where I want to deploy 500-1000 voice agents and I'm hoping that each agent would handle 10 calls in parallel, now I am self hosting this in my EC2, what would be the most optimised approach to deploy this? Should it be deployed in a docker? or python subprocess would be enough? and EC2 wise what should be my scaling approach?

Thanks for reaching out, Abhinav. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

To answer your questions: • Yes I will use livekit agent framework • My expected duration of each call is 10 minutes, it will be voice call only • Yes I prefer scaling. • I am already using python subprocess, but you would tell me why should I not use it, and there is any better approach, i am more than happy to switch. • Also I am also recording these calls using livekits tool, so consider that too

If you're looking to deploy 500-1000 LiveKit voice agents on EC2, each managing about 10 parallel calls, here’s a friendly, optimized approach to get you started: • Orchestration: Consider using AWS ECS (Elastic Container Service) or Kubernetes to keep your worker pool running smoothly. ECS works really well with EC2 setups. • Instance Types: Go for instances like

c5.xlarge

or

c4.large

that offer at least 4 vCPUs and 8 GB memory per agent—this helps keep things responsive. • Autoscaling: Set up autoscaling based on CPU and memory usage. For example, you might scale out if CPU usage stays above 75% for 5 minutes, and scale back in when it dips below 30% for 15 minutes. • Storage and Networking: Use EBS volumes to handle your logging, and make sure your VPC, subnets, and security groups are properly configured so your agents and servers can chat without a hitch. • Secrets Management: Keep your LiveKit credentials safe by storing them in AWS Secrets Manager and injecting them into your ECS tasks securely. • Monitoring: CloudWatch is your friend here—use it for health checks and to keep an eye on performance. Also, rolling updates can help you update smoothly without downtime. One nice thing is that LiveKit automatically balances the load across workers, so you don’t need an external load balancer. And if you’re open to other options, LiveKit Cloud offers built-in scaling and management for voice agents, which could make your deployment even easier. Sources: Deploying to production | LiveKit Docs | kubernetes/agent-manifest.yaml | LiveKit

I couldnt complete my previous answer, please consider these info too: To answer your questions: • Yes I will use livekit agent framework • My expected duration of each call is 10 minutes, it will be voice call only • Yes I prefer scaling. • I am already using python subprocess, but you would tell me why should I not use it, and there is any better approach, i am more than happy to switch. • Also I am also recording these calls using livekits tool, so consider that too

Hey there! Using Python's subprocess module can work fine for smaller projects or during development, but when it comes to running LiveKit voice agents in production, I’d really recommend going with Docker containers managed by AWS ECS or Kubernetes. Here’s why: • Scalability: Docker and ECS make it super easy to scale your setup up or down depending on how many agents you need—perfect for your 500-1000 agents scenario. • Consistency: Docker keeps your environment the same across development, testing, and production, which helps avoid those annoying deployment headaches. • Management Features: ECS offers handy tools like rolling updates, autoscaling, and resource management that subprocesses just don’t handle well. • LiveKit Alignment: LiveKit actually provides lots of great examples and docs for Docker-based deployments, including ECS, so you’ll have plenty of support for a smooth setup. Considering what you need, moving to Docker with ECS orchestration will give you better scalability, reliability, and keep things in line with LiveKit’s best practices. If you want, I’m here to help you get started with this setup—just let me know! Sources: Dockerfile | aws-ecs/README.md | livekit-agents/livekit/agents/voice/agent.py

Hi @alert-easter-19258 did you manage to scale up in AWS with auto scaling?

It looks like this message might have been meant for someone else, but no worries! If you have any questions or need a hand with scaling up your LiveKit deployment on AWS using auto-scaling, just let me know. I’m here to support you every step of the way!