my team and I are interested in developing a device-directed speech detection model and running it on our LK cloud server. Possibly similar in performance requirements to your turn detection model. Is this possible for us to bring our own model to run in LK cloud? Plz advise.

Thanks for reaching out, Alexander. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Can you tell me more about Livekit’s elasticity and performance mitigation requirements? I imagine LiveKit probably doesn’t want you to use up all their resources E.g. one customer has a lot of instances launched because the model they bring along takes a lot of cpu so less number of sessions per machine

You're right to think LiveKit limits resource use and isolates heavy workloads. Below is a concise summary of how LiveKit Cloud handles elasticity and performance, plus practical guidance to avoid the situation you described. Acknowledging your question • You're asking how LiveKit prevents a custom high-CPU model from consuming excessive resources and reducing session density per machine. How LiveKit manages elasticity and isolation • Automatic scaling: LiveKit Cloud will spin up new instances as demand grows and route sessions to keep performance stable. • Stateful placement: Each session is pinned to one server for the call’s duration; placement considers CPU, memory, network, and locality. • Performance isolation: High CPU usage from your agent reduces sessions on the machines running your containers but does not degrade other customers’ workloads. • Per-project limits: Concurrent agent session and deployment limits exist per plan (e.g., Build/Scale/Enterprise tiers) to prevent resource monopolization. • Warm vs cold start: Paid plans keep agents warm for faster starts; free plans may see 10–20s cold starts. Effects of high-CPU models • Fewer sessions per instance when your model is CPU-heavy. • LiveKit will scale horizontally (new instances) to accept more sessions, up to your plan’s concurrency limits. • If instances use burstable CPU types, inference can slow due to CPU throttling. Recommendations to mitigate performance issues • Optimize for CPU & memory: target CPU-friendly models or convert to smaller formats (quantize, distill, ONNX where appropriate). • Preload models at agent startup to avoid per-session cold loads. • Use compute-optimized instance types (e.g., c5/c6i or m5) — avoid burstable `t3`/`t4g` instances. • Monitor metrics (session latency, CPU) and tune model batch sizes or concurrency. • Consider splitting responsibilities: run lightweight inference in the agent and offload heavy batch jobs to separate services if possible. • Plan for higher-tier quotas or Enterprise/custom plans for very high concurrency or dedicated capacity. Next step I can help with • Review your model’s framework (PyTorch/TF/ONNX), size, and estimated CPU/RAM per inference and recommend optimizations and expected session density. Which of those details can you share? Sources: Deploy and scale agents on LiveKit Cloud | Deploy and scale agents on LiveKit Cloud | LiveKit Pricing | Quotas and limits | LiveKit docs | LiveKit turn detector plugin | LiveKit docs