So I think one issue that I'm facing right now is sometimes when deploying the worker to life get cloud, if for instance something errors out, I just need to redeploy. But when we launch in production, which is going to be very soon, we don't like, if it breaks for one user, we don't want it to break for other users and like, there should be multiple containers. containers that should keep on working.

Thanks for reaching out, Rishabh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Give me all three. It will be interesting to know what are some strategies.

Thanks — I’ll give you all three: a concise rolling-deploy checklist, a sample worker deployment config you can adapt (Kubernetes), and practical monitoring & health-check strategies with concrete metrics and triggers. I’ll keep this focused so you can act quickly. Rolling-deploy checklist (short, actionable) 1. Build & smoke-test ◦ Build your agent container image and run it locally against a staging LiveKit instance. ◦ Verify model loading, secrets, and the agent health endpoint respond correctly. 1. Prepare config ◦ Ensure all worker instances use the same

agent_name

so LiveKit can load-balance across them. ◦ Include

project.subdomain

/

agent.id

in your

livekit.toml

or env variables. 1. Staged rollout ◦ Deploy to a small subset (e.g., 1 instance of N) first. ◦ Verify the new instances accept jobs and handle sessions correctly. 1. Monitor initial traffic ◦ Watch job assignment, error logs, and latency for the new instances for a short window (5–15 minutes). 1. Roll forward gradually ◦ Increase new-version replicas in waves (e.g., 25% → 50% → 100%) while keeping old replicas available. 1. Drain old workers gracefully ◦ Allow old workers to finish active sessions; do not force-kill. Use termination grace periods. 1. Validate and finalize ◦ Confirm new workers handle normal load. If issues appear, run an immediate rollback. 1. Rollback plan ◦ Keep the previous working version ready. Roll back instantly if errors spike. Sample Kubernetes deployment (you can paste and adapt)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: livekit-agent-worker
spec:
  replicas: 3
  selector:
    matchLabels:
      app: livekit-agent
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: livekit-agent
    spec:
      containers:
        - name: agent
          image: your-agent-image:latest
          env:
            - name: AGENT_NAME
              value: "your-agent-name"
            - name: PROJECT_SUBDOMAIN
              valueFrom:
                secretKeyRef:
                  name: livekit-secrets
                  key: project_subdomain
          resources:
            requests:
              cpu: "4000m"
              memory: "8Gi"
            limits:
              cpu: "4000m"
              memory: "8Gi"
          livenessProbe:
            httpGet:
              path: /health
              port: 8081
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /health
              port: 8081
            initialDelaySeconds: 10
            periodSeconds: 5
      terminationGracePeriodSeconds: 600

Key config notes: • replicas: start with 3 for redundancy; increase for higher capacity. • readiness/liveness: point to the agent’s /health endpoint so Kubernetes only routes traffic to healthy pods. • terminationGracePeriodSeconds: set long enough (e.g., 600s) to allow sessions to finish gracefully during rolling updates. • CPU/memory: tune per your agent workload; the example uses 4 CPU / 8GB as a baseline for voice/LLM agents. Monitoring & health-check strategies (what to watch and how to react) • Health endpoints ◦ Use the agent’s

/health

for Kubernetes liveness/readiness probes. ◦ Readiness should reflect “can accept new jobs”; liveness should reflect process health. • Auto-drain & load protection ◦ Ensure workers expose a load metric (or respect a

load_threshold

) so they stop taking new jobs before saturation. • Metrics to collect ◦ Session/job counts per worker (active sessions). ◦ CPU and memory utilization (per pod). ◦ Job error/crash rate (exceptions per minute). ◦ Latency metrics: Time To First Token (TTFT) and Time To First Byte (TTFB) for LLM/audio responses. ◦ Connection-quality signals (packet loss, jitter) if available. • Alerts & thresholds (examples) ◦ CPU > 70% sustained for 1–2 minutes → scale up. ◦ Error rate spike > 3x baseline → roll back or divert traffic. ◦ Increase in TTFT/TTFB > 2x baseline → investigate model or network issues. ◦ Pod restarts > 3 times in 5 minutes → mark node unhealthy and remove from rotation. • Logs & tracing ◦ Stream

lk agent logs

during rollout for build/deploy errors. ◦ Capture worker crash stacks and the associated session ID to replay/debug in staging. • Dashboard & CLI checks ◦ Use LiveKit Cloud dashboard for realtime session counts, agent status, and deployment logs. ◦ CLI commands to use: ◦

lk agent deploy

to deploy ◦

lk agent status --id <agent-id>

to check rollout ◦

lk agent logs --id <agent-id>

to tail logs ◦

lk agent rollback --version <previous>

to revert quickly • Autoscaling recommendations ◦ Scale by CPU or job-queue depth. Start with CPU 50% as a scale-up trigger to maintain headroom. ◦ Maintain at least 2–3 instances during low traffic to avoid single-failure impact. • Runbook actions for failures a. If a single pod crashes: let orchestrator restart it; check crash logs. If repeated, cordon node and replace image/version. b. If new version causes errors across pods: initiate immediate rollback to the previous version. c. If capacity shortages cause “room connection not established within 10s”: scale up replicas and prioritize urgent users if supported. Quick deployment / reliability strategies (practical tips) • Staggered deployments: never replace all workers at once — deploy a canary, validate, then continue. • Warm instances: keep a small pool of warm workers to avoid cold-start latency for realtime sessions. • Isolation of failures: keep agent workers dedicated (no other heavy processes on the same nodes). • Secrets & config updates: treat secret changes as rolling restarts; plan for them in deploy windows. If you want, I can: • Provide a short, tailored rolling-deploy checklist based on your current cluster size and CI/CD pipeline. • Generate a Kubernetes manifest tuned to your expected concurrent sessions (tell me expected concurrent rooms & average session duration). • Recommend specific alert thresholds if you share typical baseline CPU, memory, and error rates. Which of those would you like next? Sources: Examples | livekit/livekit_composite | DeepWiki | Agents (Node.js) | livekit/livekit_composite | DeepWiki | Agents (Python) | livekit/livekit_composite | DeepWiki | Server-side | livekit/livekit_composite | DeepWiki | Deploy and scale agents on LiveKit Cloud

Can you help understand the costing?

Thanks — I’ll walk you through costs clearly and help you estimate what your production run might cost. I’ll first summarize the pricing components that matter for agent worker deployments, then show a simple worked example you can adapt. Finally, I’ll ask a few quick questions so I can give a precise estimate for your usage. What drives cost (short summary) • Agent session minutes: $0.01 per active agent-minute. This covers hosting, data transfer to/from the agent, and observability. • Concurrency limits: Each plan has a concurrent-agent cap (Build: 5, Ship: 20, Scale: 600+, Enterprise: unlimited). Pick a plan that matches your peak concurrent sessions. • Inference credits / model costs: STT, TTS, and LLM inference are billed separately (e.g., per minute of audio for STT, per character for TTS, per million tokens for LLM). Model choice heavily impacts cost. • Human participant costs: Participant minutes and downstream bandwidth incur small additional fees (participant connection fee and $/GB downstream). • Telephony (optional): Phone numbers and inbound minutes are billed if you use PSTN/SIP. • Plan included allowances: Each plan includes free agent minutes and some inference credits; beyond that you pay per-use. Quick example estimates (illustrative) • Single 10-minute voice session with a deployed agent: ◦ Agent session: 10 min × $0.01 = $0.10 ◦ Inbound telephony (if used): 10 min × $0.01 = $0.10 ◦ STT/TTS/LLM: depends on model — rough additional range ~$0.08–$0.20 for moderate LLM/stt usage ◦ Total per session (rough): $0.28–$0.40 • For 1,000 such sessions/month (10 min each): 10,000 agent-minutes → $100 in agent session minutes + inference/telephony on top. How concurrency and autoscaling affect cost • Agent session minutes are the main usage driver (not number of pods). Running multiple workers increases capacity and availability but you pay mainly for the minutes agents are actively serving users. • Keep spare capacity (≥2–3 workers) to avoid single-point failures; that adds small overhead only when those workers are actively used or kept warm (cold-start prevention may keep agents warm and increase billed minutes). What I need to give you a precise estimate Please share these quick numbers (or approximate): 1. Expected average session length (minutes). 2. Expected number of sessions per day or total agent-minutes per month. 3. Peak concurrent sessions (to pick a plan). 4. Whether you’ll use telephony (PSTN/SIP). 5. Which LLM/STT/TTS models you expect to use or a rough per-session inference load (e.g., “short prompts, small responses” vs “long prompts, long generated responses”). If you prefer, tell me: • Monthly agent-minutes you expect (if known), and I’ll convert that into cost using $0.01/min + a simple inference estimate per minute. Extra help I can provide • A cost spreadsheet template you can plug numbers into. • A sample calculation tuned to your expected model usage (I can use conservative, typical, and aggressive inference assumptions). • Recommendations on which plan to pick given your peak concurrency and monthly minutes. Which would you like next — give me those 5 numbers above, or tell me monthly agent-minutes and model usage and I’ll produce a concrete cost estimate? Sources: Deploy and scale agents on LiveKit Cloud | Pricing | LiveKit | Build voice, video, and physical AI | LiveKit | Towards a future-aligned pricing model

recalculate based on the following: no telephony.

Great — I’ll recalculate cost estimates with telephony removed. I’ll summarize the pricing components that remain, then show a worked example and ask one quick question if I need any missing inputs. What changes when telephony is removed • Remove PSTN/SIP line items entirely (phone numbers and inbound-minute charges no longer apply). • Remaining cost drivers are: ◦ Agent session minutes: $0.01 per active agent-minute. ◦ Inference costs for STT, TTS, and LLM usage (billed separately via inference credits). ◦ Human participant connection and downstream bandwidth charges (if you have human participants in rooms). ◦ Any plan subscription fees and included allowances (free agent minutes and inference credits per plan). Assumptions I’ll use for the worked examples below • No telephony. • An example session is a voice-only agent session (user speaks; agent responds via TTS). • Agent session rate: $0.01/min. • Inference: I’ll provide three inference scenarios (conservative / typical / heavy). If you tell me a specific model, I can refine those numbers. Worked examples (per 10-minute session) 1) Conservative inference (small prompts, small responses) • Agent session: 10 min × $0.01 = $0.10 • STT: ~10 min × small per-minute STT cost ≈ $0.02 • LLM: small prompt + small response ≈ $0.01 • TTS: small output ≈ $0.005 • Total ≈ $0.135 per 10-minute session → $0.0135 per minute 2) Typical inference (moderate prompt + response) • Agent session: $0.10 • STT: ≈ $0.03 • LLM: moderate usage ≈ $0.05 • TTS: ≈ $0.01 • Total ≈ $0.19 per 10-minute session → $0.019 per minute 3) Heavy inference (long prompts, long responses / large LLM) • Agent session: $0.10 • STT: ≈ $0.05 • LLM: heavy usage (e.g., lengthy generations) ≈ $0.40 • TTS: ≈ $0.02 • Total ≈ $0.57 per 10-minute session → $0.057 per minute Monthly examples (scale-up) • If you run 10,000 agent-minutes/month (e.g., 1,000 sessions × 10 min): ◦ Agent session minutes: 10,000 × $0.01 = $100/month ◦ Inference add-on: ▪︎ Conservative: ≈ $13.50 → Total ≈ $113.50 ▪︎ Typical: ≈ $19.00 → Total ≈ $119.00 ▪︎ Heavy: ≈ $57.00 → Total ≈ $157.00 Notes about plans and included allowances • Each plan includes free agent minutes (Build: 1,000; Ship: 5,000; Scale: 50,000). Use those first — only pay overage at $0.01/min. • Inference credits included per plan vary (Build small, Ship moderate, Scale larger). If your inference needs are significant, the Scale plan provides discounted model rates and larger included credits. Human participant and bandwidth considerations • If human participants join rooms, participant-minute and downstream bandwidth charges still apply: ◦ Participant connection fee (small per-minute). ◦ Downstream bandwidth: $0.12/GB (upstream is free). • If sessions are agent-only (no humans), you can ignore these. What I need from you to make a precise estimate Please tell me any of the following you already know: 1. Expected average session length (minutes). 2. Expected total sessions per month or total agent-minutes per month. 3. Peak concurrent sessions (to choose an appropriate plan). 4. Which LLM/STT/TTS models (or model classes) you plan to use, or an estimate of average LLM tokens per session / STT minutes / TTS characters. 5. Whether human participants will join sessions (and if so, approximate participant-minutes and downstream GB/month). Next steps I can take for you • Produce a detailed monthly cost spreadsheet using your inputs above. • Recommend which LiveKit plan likely fits your concurrency and minutes needs. • Provide cost-optimization tips (model selection, batching prompts, reducing TTS verbosity). Which would you like me to do next? If you share 1–3 above (session length, sessions/month, peak concurrency), I’ll produce a concrete monthly cost estimate with plan suggestions.