How does LiveKit determine the capacity of every worker when applying the round robin?

Thanks for reaching out, Tarek-Asis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I have a custom load_fnc that uses the concurrent jobs per worker so we know the maximum number of jobs a worker can handle. The problem is that the workload is not evenly distrubuted

Thanks — I hear you: you’re using a custom

load_fnc

based on concurrent jobs but workload is uneven. A few quick questions to pinpoint the cause so I can give targeted suggestions: • How does your

load_fnc

report load? (e.g., load = active_jobs / max_jobs, or just active_jobs) • Is

load_threshold

set, and what value are you using? • Are all workers identical (CPU/memory/network) and registered with the same max_jobs capacity? • Do jobs vary in resource cost or duration, or are they roughly equal? • Are you seeing any job acceptance timeouts or reassignments in logs? • Are you using geographic affinity in LiveKit Cloud or any custom routing rules? Answering these will let me recommend whether to adjust the metric (use normalized load), add smoothing/averaging, include CPU/memory signals, or tweak thresholds/timeouts. , with interactive elements

1. load = _min_(active / MAX_CONCURRENT_JOBS, 1.0) 2. load_threshold = 1.0 3. Yes 4. Yes, depends on the type of call 5. Nope 6. Nope

Thanks — I’m responding to your uneven distribution while using

load = min(active / MAX_CONCURRENT_JOBS, 1.0)

and

load_threshold = 1.0

. Quick diagnosis and actionable fixes: • Problem root: Using raw concurrent-job fraction with threshold 1.0 lets workers hit full capacity before being excluded. If jobs vary in cost/duration, workers handling heavier jobs still count the same as light ones, causing imbalance. Actions to try (pick 1–2 to iterate): 1. Normalize by effective job weight ◦ Assign each job a weight (e.g., 1.0 for light, 2.5 for heavy). Report load = min(sum(weights) / MAX_CAPACITY, 1.0). ◦ MAX_CAPACITY can be tuned per worker if hardware differs. 1. Lower the load_threshold ◦ Set

load_threshold

below 1.0 (start at 0.6–0.8) so dispatch avoids saturating workers and evens distribution. 1. Smooth/average load signal ◦ Use a short moving average (5–15s) for load so transient spikes don’t bias assignments. 1. Include a lightweight resource signal ◦ Combine active-job fraction with CPU or memory (e.g., load = 0.6 * job_fraction + 0.4 * cpu_fraction) to reflect real capacity. 1. Prefer least-loaded when equal ◦ If your dispatcher supports tie-breaking, prefer workers with lower recent average load rather than strict round-robin. 1. Use weighted max_concurrent per worker ◦ If some workers handle heavy calls more efficiently, set different MAX_CONCURRENT_JOBS to reflect true capacity. If you want, tell me: • the typical weight ratio heavy:light (e.g., heavy ≈ 3x light) and • whether you can emit CPU metrics and I’ll propose the exact

load_fnc

formula and sample parameters to test. , with interactive elements

But the problem here is that almost all the load were being sent to the same worker

Thanks — I hear you: most traffic is ending up on a single worker even though you’re using

load = min(active / MAX_CONCURRENT_JOBS, 1.0)

. Let’s narrow the likely causes and give quick checks and fixes. Quick checks (please run these and share results if you want deeper help): • Confirm the dispatcher’s tie-break behavior: does it prefer the last-used worker or truly rotate when loads are equal? If tie-breaking favors a single worker, that explains the behavior. • Verify the per-worker reported load values over a short window (5–30s). Are other workers reporting lower or equal load when the hot worker is chosen? • Ensure MAX_CONCURRENT_JOBS is set correctly and identically on every worker (no misconfig causing one worker to be treated as having much higher capacity). • Check whether jobs of different types are routed deterministically (e.g., same caller ID, room affinity, or geographic routing) that pins sessions to one worker. Immediate actionable fixes to test (pick 1–2): 1. Lower

load_threshold

from 1.0 to 0.6–0.8 so the dispatcher avoids saturating a single worker. 2. Add tie-breaker logic: prefer the worker with the lowest recent average load rather than strict round-robin order. 3. Smooth the load signal with a short moving average (5–15s) so transient dips don’t bias selection. 4. Use weighted job counts: report load as sum(job_weight)/MAX_CAPACITY so heavy jobs reduce a worker’s available capacity more. 5. If deterministic routing rules exist, disable or relax them temporarily to confirm the dispatcher is the cause. If you’d like, share: • a short sample of per-worker reported loads during a skewed period (timestamps + load values), and • the dispatcher/tie-break configuration and I’ll suggest the exact

load_fnc

tweak and tie-break rule to fix the single-worker hotspot. , with interactive elements

How can I add tie-breaker logic? Is not it part of the LiveKit logic?

You're right to ask — LiveKit's dispatcher uses a load-aware round-robin style but does not provide a rich, configurable tie-breaker out of the box. When multiple workers report equal eligibility, the default behavior can effectively favor a single worker (round-robin pointer or last-used behavior), which can produce the hotspot you're seeing. Two ways to add tie-breaker behavior: • Change

load_fnc

to encode tie-breaking (recommended, no dispatcher code changes needed) ◦ Add a small penalty to the load reported by the most-recently-used worker so it looks slightly busier: ▪︎ Example approach: maintain a short history of recently-assigned worker IDs on the dispatcher. When computing

load

, if worker_id == last_assigned, report load = min(actual_load + 0.01, 1.0). ▪︎ For stronger effect, increase penalty (0.05–0.2) or apply exponentially decaying penalties for the last N assignments. ◦ Or encode a tiny jitter so equal loads rarely tie: reported_load = min(actual_load + random(0, 0.001), 1.0). • Modify dispatcher selection logic (self-hosted only) ◦ If you control dispatcher code, change the eligible-worker selection to: i. Filter workers with load < load_threshold. ii. Sort by (recent_average_load, recent_assignment_count, latency) ascending. iii. Select the first worker (or pick randomly among the lowest N). ◦ Prefer deterministic policies like “least recent assignment” or “least recent average load,” or randomized selection among the lowest-load set. Which approach is best: • If you cannot modify the dispatcher, implement the tie-breaker in

load_fnc

(penalty or jitter). This is simplest and effective. • If you self-host and can change the dispatcher, implementing a true least-loaded / least-recent-assignment tie-breaker gives the most control. If you want, tell me: • whether you can modify the dispatcher code or only the

load_fnc

, and • whether you prefer a deterministic tie-breaker (least-recently-used) or randomized tie-breaking, and I’ll provide a concrete

load_fnc

snippet or dispatcher selection pseudocode to implement it. , with interactive elements