when I use livekit cloud all my calls go through instantly with the allowed concurrency, but when i switch to my self hosted livekit server I see dispatch failing. I had added retry to make sure the call goes through but its creating a huge bottleneck for my project. How to solve this?

Thanks for reaching out, Aditya. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

but how is it working with the livekit-cloud?

Great question — and a good one to understand why behavior differs between Cloud and self-hosted. Why LiveKit Cloud dispatches succeed instantly • Built-in autoscaling: LiveKit Cloud automatically scales agent capacity up and down to match incoming load. When concurrency spikes, more workers are provisioned immediately so dispatches find available agents. • Globally distributed infrastructure: Cloud runs servers and agents across regions and load-balanced endpoints, so dispatches are routed to healthy, nearby workers with low latency. • Managed agent pool health: Cloud maintains healthy persistent connections from agents and performs service-level health checks and routing, so jobs aren’t sent to nodes that are overloaded or disconnected. • Tuned defaults: Cloud uses tuned thresholds and load‑measurement logic so agents won’t reject work unnecessarily; combined with autoscaling this prevents the “no worker available” failures that trigger client retries. Why a typical self-hosted deployment can fail where Cloud doesn’t • Fixed capacity: If your self-hosted agent pool is too small, dispatches will fail when all workers are busy — Cloud would have already scaled out. • No global routing: Self-hosted setups often run in one region or a smaller footprint, so latency or network issues are more likely to disrupt agent connections. • Default load thresholds: Self-hosted AgentServer defaults (for example

load_threshold

) may be conservative for your workload, leading to rejection of new jobs when you still expect capacity. • Connectivity/ops differences: Maintaining persistent WebSocket connections, handling health checks, and implementing graceful scaling are responsibilities you must cover in self-hosting — Cloud manages those for you. What you can do to get Cloud-like behavior on your self-hosted setup • Autoscale your agents proactively (scale sooner than the rejection threshold). • Run multiple worker processes/containers per

agent_name

. • Tune

load_threshold

and

load_fnc

to reflect your environment. • Ensure stable, persistent network connections between agents and the server. • Spread agents across regions if you have geographically dispersed users. • Replace tight retry loops with exponential backoff + jitter and a capped max attempts. If you’d like, I can recommend specific autoscaling thresholds and a retry strategy tailored to your current agent counts and traffic pattern. Share one of these and I’ll give concrete settings: • Current number of agent workers and regions • Your

load_threshold

or AgentServer config • Example dispatch failure log lines you’re seeing Which would you like to start with?

there is no dispatch failure, it just times out.

Thanks — that clarification helps a lot: you’re seeing dispatch requests time out on your self-hosted server (not hard failures). I’ll explain the most likely causes and give focused steps to stop those timeouts so retries stop creating the bottleneck. Why LiveKit Cloud doesn’t time out (short summary) • Cloud auto-scales agents and keeps a healthy, globally distributed agent pool, so jobs find workers quickly and dispatch latency stays low. Why self-hosted dispatch can time out • Resource constraints on agent servers (CPU/memory) can make agents slow to accept work. • Agents that are near or above the

load_threshold

stop accepting new jobs, which increases wait time until a worker is available. • Network issues or unstable WebSocket connections between agents and the LiveKit server delay assignment. • Single-region or under-provisioned agent topology increases latency versus Cloud’s distributed routing. • Default dispatch-related time windows can be exceeded (participants not joining or jobs not accepted within the dispatch timeout). Concrete checks and fixes (priority order) 1. Verify agent capacity ◦ Ensure you run multiple agent worker processes/containers per

agent_name

. Single instances cause queueing and timeouts. ◦ Recommended baseline for typical voice-AI workloads: at least 4 CPU cores and 8GB RAM per agent server. Under-provisioning causes slow acceptance. 1. Observe and tune load behavior ◦ Agents reject new jobs when their

load_threshold

(default 0.7) is exceeded. That leads to longer assignment times. ◦ Tune

load_threshold

and implement a

load_fnc

that matches your environment (CPU, queue depth, or job count) so you detect and scale before timed-out dispatches occur. 1. Autoscale earlier and proactively ◦ Scale out at a lower trigger than the rejection threshold (for example, scale when CPU ~50%, not 70%). ◦ Make scaling fast enough so new workers are ready before traffic peaks. 1. Stabilize connectivity ◦ Confirm persistent WebSocket connections from agents to the LiveKit server. Check for firewall/NAT interruptions or frequent reconnects. ◦ If agents are briefly disconnected, they can’t accept jobs and dispatch requests will timeout. 1. Reduce retry pressure ◦ Replace tight retry loops with exponential backoff + jitter and a capped max attempts. Immediate retries during saturation amplify timeouts. ◦ Prefer to fix capacity/connectivity rather than rely on aggressive retries. 1. Add logging and metrics to pinpoint the timeout cause ◦ Capture timestamps and log lines for: dispatch request sent, job assigned, agent accepted, participant joined/connected. ◦ Key metrics: agent connection counts, CPU usage, rejection counts due to

load_threshold

, dispatch latency, and retry counts. 1. Consider geographic affinity ◦ If users are far from your agents, deploy agent servers closer to users to reduce network latency and dispatch time. What I can do next to give a targeted plan If you share any of the following, I’ll propose exact settings and autoscaling recommendations: • Number of agent worker instances and their CPU/RAM per region • Your

load_threshold

and

AgentServer

config (if customized) • Example log entries around a timeout (dispatch request timestamp + any agent-side logs) If you prefer, send one small item (e.g., current agent count) and I’ll give a concrete autoscaling threshold and retry-backoff configuration you can apply immediately. Sources: Agent dispatch | LiveKit Documentation | Self-hosted deployments | LiveKit Documentation | Server options | LiveKit Documentation | Agents Framework | livekit/livekit_composite | DeepWiki | Agents telephony integration | LiveKit Documentation

i have livekit-agents server separately where the agent sessions happen, now I was using livekit self hosted as a separate service with the SIP server and Redis. Recently I started to get issues where the my agent-server sends SIP dispatch request to self-hosted livekit server which is connected via Websoket_URL and API key and API secret, and I am observing a huge chunk of outbound calls dropping. but when I am using the livekit clouds WSS url and API keys they all are instantly called?

Thanks — that’s very helpful detail. I understand: your livekit-agent servers (handling SIP sessions) send dispatch requests to your self‑hosted LiveKit instance over the WebSocket_URL using an API key/secret, and you see a large number of outbound calls drop or time out. The exact same agent setup works instantly when pointed at LiveKit Cloud’s WSS URL and API keys. I’ll explain the most likely causes and give focused, prioritized troubleshooting steps you can run quickly. What’s different between Cloud and your self-hosted server • LiveKit Cloud keeps a healthy, autoscaled, globally distributed agent pool and tuned routing, so dispatches reach available workers quickly. • Your self-hosted control plane now needs to provide the same reliability: persistent WebSocket connectivity from agents, sufficient agent capacity, correct TLS/HTTP routing, and correct configuration for SIP/Redis interactions. If any of these are weak, dispatch requests can time out or get dropped. Most likely causes for outbound SIP calls dropping on self-hosted • Network/TLS/connectivity problems between agent servers and your LiveKit self-hosted WebSocket endpoint (firewall, NAT timeout, proxy, or TLS mismatch). • Agent WebSocket connections intermittently disconnecting or not staying established, so agents miss dispatches. • Under-provisioned agent servers (CPU/RAM) or too few worker processes causing slow acceptance of jobs and timeouts. • Load thresholds: agents or the LiveKit server rejecting or delaying dispatch assignment once load exceeds

load_threshold

. • Redis or backend coordination issues (if your deployment uses Redis for state/coordination) causing delayed job routing. • Misconfigured SIP timeout or telephony integration settings that delay participant creation, leading to dispatch timeouts. • Web server / reverse-proxy misconfiguration in front of the LiveKit WebSocket endpoint (incorrect timeouts, missing WebSocket upgrade handling, or buffering causing delays). Immediate checks to run (quick, high-value) 1. Confirm persistent WebSocket connectivity ◦ On an agent host, check the agent’s WebSocket connection state: is it connected continuously or showing frequent reconnects? ◦ Check reverse-proxy (nginx/envoy/traefik) config to ensure WebSocket upgrades and idle timeouts are long enough (no short proxy timeouts killing the connections). 1. Inspect TLS and endpoint correctness ◦ Verify the self-hosted WSS URL is exactly the address the agent uses and the certificate is valid and trusted on the agent host. ◦ Compare headers and TLS versions between Cloud WSS (working) and self-hosted WSS (failing) to spot differences. 1. Check agent resource usage and worker count ◦ Monitor CPU and memory on your agent servers during peaks. Are they spiking, causing slower job acceptance? ◦ Ensure multiple agent worker processes/containers are running for each

agent_name

to allow concurrent dispatches. 1. Review load and rejection metrics ◦ Look for logs/metrics showing agents rejecting jobs due to

load_threshold

or custom

load_fnc

behavior. ◦ If rejections are happening, lower the autoscale trigger or

load_threshold

tuning and scale out sooner. 1. Watch for Redis/coordination errors ◦ Check Redis latency/errors if you use it for state. High Redis latency can delay job routing and lead to timeouts. 1. Examine server and agent logs around a failure ◦ Capture a timeline: when the agent sends dispatch → LiveKit receives dispatch → LiveKit forwards/assigns → agent accepts and SIP participant created. ◦ Note any logged delay or error at each step (timestamps help). Short-term mitigations to reduce drops while you investigate • Use exponential backoff + jitter and a low capped retry count instead of tight immediate retries to avoid making the overload worse. • Temporarily increase dispatch timeouts (if configurable) to tolerate slightly longer assignment times while you scale capacity. • Scale agent workers up (add instances/processes) to absorb current traffic spikes. Targeted tests to isolate root cause • Point a single agent instance to the self-hosted WSS and reproduce a call while tailing logs on both agent and LiveKit server to see where the delay appears. • Temporarily route an agent through the same network path as Cloud (if possible) to test whether network path / firewall is the difference. • Swap in a simple WebSocket health check client that maintains a connection to the self-hosted WSS and logs disconnects/reconnects over time. What I can do next If you share one or more of the following, I’ll give concrete next steps and exact settings to try: • The WebSocket_URL you’re using (or a description of the reverse proxy in front of it), and whether you terminate TLS at the proxy • Example log snippets from an agent and from the LiveKit server for a call that timed out (timestamps) • Agent server CPU/RAM and number of worker processes during a spike • Any proxy (nginx/envoy) timeouts and config around WebSocket handling Which of those can you share now? Even the agent log for a single failed call will let me point to where the handoff is breaking and propose an immediate fix.