How does livekit worker handles load? I deployed them on aws ecs and isee the cpu usage around 50% yet the workers won’t accept new jobs. Should I scale out based on aws reported cpu usage or a custom livekit metric?

Thanks for reaching out, Abhishek. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i’m not using load_fnc and load_threshold yet. How do I scale out based on these params? Does livekit worker send these thresholds to aws so that they can be used for scale-out?

Thanks — you're asking how to use

load_fnc

/

load_threshold

to scale out, and whether LiveKit pushes those thresholds to AWS. Short answers: • You configure

load_fnc

and

load_threshold

on the LiveKit worker; LiveKit does not automatically send those thresholds to AWS. • For autoscaling you should expose/use the same metric that your

load_fnc

returns (CPU by default) and make your ECS scale policy act at a lower threshold than

load_threshold

so new instances are added before workers start rejecting jobs. What to do next (recommended steps): 1. Configure

load_fnc

and

load_threshold

in your worker process. If you keep defaults,

load_fnc

= overall CPU and

load_threshold

= 0.7. 2. Export the same metric to AWS/ECS: ◦ If

load_fnc

is CPU, use ECS/CloudWatch CPUUtilization. ◦ If you implement a custom

load_fnc

(e.g., combining CPU, memory, network, or a Prometheus gauge), publish that value to CloudWatch as a custom metric (or use CloudWatch agent / Prometheus -> CloudWatch). 1. Create ECS autoscaling policies based on that metric: ◦ Scale up when the metric reaches a value below

load_threshold

(example: if

load_threshold = 0.7

, scale up at ~0.5). ◦ Scale down with a lower threshold and include a cooldown/drain period to allow existing sessions to finish. 1. Ensure graceful draining on scale-in so stateful sessions are preserved (stop accepting new jobs on the instance, wait for active sessions to finish or migrate). If you want, tell me: • which metric you plan to use for

load_fnc

(default CPU or a custom combination), and • whether you use CloudWatch or a Prometheus pipeline — I’ll suggest specific CloudWatch metric/threshold settings and a sample autoscaling policy., with interactive elements

default cpu is fine. I use cloudwatch

Great — since you’re using the default CPU metric and CloudWatch, here’s a concise plan to make autoscaling align with LiveKit’s load behavior. Summary: • LiveKit does not push

load_threshold

to AWS. You must base ECS autoscaling on the same metric LiveKit uses (

CPUUtilization

by default). • Default LiveKit behavior:

load_fnc

= overall CPU,

load_threshold

= 0.7 (worker stops accepting new jobs when load ≥ 0.7). Recommended steps: 1. Ensure your workers use the default

load_fnc

/

load_threshold

(or set them explicitly if needed). 2. In CloudWatch, use the ECS/EC2

CPUUtilization

metric for the tasks or instances running LiveKit. 3. Create scale-up and scale-down policies: ◦ Scale up when CPUUtilization reaches ~50% (0.5). This is below

load_threshold

so new capacity arrives before workers reject jobs. ◦ Scale down when CPUUtilization drops below ~30% (0.3), and use a cooldown period. 1. Configure cooldowns and draining: ◦ Set a scale-down cooldown (e.g., 5–15 minutes) to avoid thrash. ◦ Enable ECS draining on scale-in so tasks stop accepting new sessions and existing sessions can finish. Wait for active sessions to complete or implement a graceful drain timeout. 1. Monitor and tune: ◦ Watch actual session durations and CPU patterns; adjust scale-up (0.45–0.6) and scale-down (0.25–0.35) thresholds if needed. ◦ If you see sustained high CPU but AWS shows ~50%, verify task-level vs. instance-level CPU metrics and use the metric that corresponds to where

load_fnc

runs (task CPU if worker runs in a task). If you want, I can: • Suggest exact CloudWatch alarm settings and an ECS scaling policy JSON, or • Recommend drain-time values based on your typical session length — tell me average session duration., with interactive elements

how frequenctly load_fnc method called?

You're asking how often

load_fnc

is sampled. Quick answer: • The default

load_fnc

uses the agent server's average CPU utilization over a 5-second window. • The docs do not document a configurable polling interval for

load_fnc

. • You can implement a custom

load_fnc

(e.g., different sampling window or combining CPU/memory/network) if you need a different cadence; if you do, publish that same value to CloudWatch and base ECS scaling on it. Do you want a short example of a custom

load_fnc

and how to push it to CloudWatch? Sources: Server options | LiveKit Documentation | Self-hosted deployments | LiveKit Documentation | Agents on LiveKit Cloud | Deploy voice AI on global infrastructure | LiveKi..., with interactive elements