in self hosting livekit server on a vm this error ...
# ask-ai
p
in self hosting livekit server on a vm this error occur on the sip server "jitterBuf": false, "status": 486, "reason": "flood", "stats": {"port": and the agent suddenly stopped accepting the call what is the cause of the issue and how to fix it
t
Thanks for reaching out, Yasser. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — this helps. I’m responding to the SIP error you shared (SIP response 486 with "reason": "flood") and the sudden stop in the agent accepting calls. Short answer: a SIP 486 “Busy Here” with a reason of "flood" means the SIP endpoint or proxy thinks it is being overwhelmed by too many requests (in your case likely INVITEs or re-INVITEs). That causes calls to be rejected and the agent to stop accepting further calls. To pinpoint and fix it I need a few details, and I’ll list the most likely causes and concrete steps to diagnose and resolve the issue. Please provide: • LiveKit server version and SIP gateway component/version you’re running. • Whether you’re using LiveKit’s built-in SIP gateway or an external SIP proxy (and its config). • Approximate call/transaction rate when the problem occurs (calls per second or concurrent calls). • VM resources (CPU, RAM, NIC load) and whether resource usage spikes when the flood occurs. • Relevant logs/snippets around the error (SIP trace or LiveKit/SIP gateway logs) — especially the lines immediately before the 486/flood message. Likely causes and how to check/fix them • High request rate (real flood or burst): ◦ Cause: Many INVITEs/re-INVITEs or re-registrations arrive in a short time; SIP proxy or SIP endpoint treats it as a flood and responds 486. ◦ Check: Review SIP logs/pcap to see how many requests per second arrive and which IPs send them. ◦ Fixes: ▪︎ Rate-limit or throttle upstream callers or the SIP proxy. ▪︎ Add or tune anti-flood settings in the SIP proxy/gateway so legitimate bursts aren’t blocked. ▪︎ If calls are legitimate, scale horizontally (add more SIP gateway workers) or increase VM capacity. • Retransmissions / network instability causing duplicates: ◦ Cause: Packet loss or high latency causes retransmissions that look like a flood. ◦ Check: Capture a short pcap (tcpdump) on the VM NIC during the event and look for retransmissions or repeated INVITEs. ◦ Fixes: ▪︎ Fix network issues (packet loss, high jitter). ▪︎ Tune SIP retransmission timers or adjust NAT/ALGs on the network path. • Misconfigured SIP proxy / aggressive anti-flood rules: ◦ Cause: Proxy or gateway rules classify normal activity as flood (e.g., low threshold). ◦ Check: Inspect the SIP proxy/gateway flood/DoS protection settings. ◦ Fixes: ▪︎ Raise thresholds for rate-limits temporarily while you tune. ▪︎ Whitelist trusted upstream SIP peers or internal components. • Resource exhaustion on the VM / SIP gateway crash or thread starvation: ◦ Cause: CPU or memory saturated so the SIP agent stops processing calls; proxy responds 486. ◦ Check: Monitor CPU, memory, and process counts on the VM at the time of the error (top, vmstat). ◦ Fixes: ▪︎ Increase VM resources or optimize worker concurrency. ▪︎ Tune LiveKit/SIP gateway worker/thread counts to match available cores. • Session handling bug or unexpected state in the agent: ◦ Cause: Agent enters a bad state and refuses new INVITEs. ◦ Check: Review gateway and LiveKit logs for errors/exceptions preceding the 486/flood message. ◦ Fixes: ▪︎ Restart the SIP gateway/agent to restore normal state while you investigate logs. ▪︎ If reproducible, gather detailed logs and we can triage further. Immediate diagnostic steps you can run now 1. Capture SIP logs and a short packet capture around the time the 486 occurs: ◦ Save the SIP gateway logs and a pcap of the VM NIC for ~30–60 seconds when issue reproduces. 1. Check VM resource metrics at the same time: ◦ CPU, memory, disk I/O, and network interface stats. 1. Search logs for repeated INVITE or REGISTER entries from the same source IP right before the 486. 2. Temporarily restart the SIP gateway process to restore call handling while you gather log evidence. If you can share the requested details and a short log/pcap excerpt, I’ll help interpret them and suggest exact config changes (rate limits, worker counts, or other tunings). If you prefer, tell me which SIP component you want to start with (LiveKit SIP gateway or an external proxy) and I’ll provide targeted commands and config examples.
p
does the root cause coming from livekit side or from the call center? explain in details the call center route all its calls to livekit server then livekit agent answer the calls
t
Thanks — I’ll explain clearly which side is responsible, why that 486/"flood" response happens, and what to check on both LiveKit and the call center so you can pinpoint and fix it. Short answer • The 486 "Busy Here" with reason "flood" is coming from the LiveKit side (the SIP gateway/agent) — it’s signaling that the LiveKit agent or gateway is refusing calls because it believes it’s being overwhelmed or is unavailable. The call center is correctly routing calls to your LiveKit server; the decision to accept or reject is being made by your LiveKit components. Why that happens (detailed) • Agent availability and dispatch behavior: ◦ LiveKit Agents act as the endpoint that accepts dispatched jobs (incoming calls). If an agent is busy, crashed, or explicitly rejects a job, LiveKit will respond that the agent is unavailable (resulting in a 486). ◦ Some configurations or versions stop dispatching jobs after a rejection (so subsequent calls are not routed to other agents) — this will make it appear like agents “suddenly stop accepting calls.” • “Flood” reason meaning: ◦ The "flood" tag indicates the SIP gateway or agent believes it is receiving too many requests in a short time (INVITEs, re-INVITEs, or other SIP transactions) or sees duplicate/rapid requests that look like a flood/DoS. ◦ That can be caused by legitimate high call volume, retransmissions/repeats due to network issues, or overly aggressive anti-flood thresholds on the SIP gateway. • Resource exhaustion or process state: ◦ If the SIP gateway or the agent process is starved for CPU, memory, or threads, it may stop processing new calls and start rejecting them. ◦ A buggy agent implementation or a state where the agent never returns to a ready state after an error can also cause repeated rejections. What to check now (practical diagnostics) 1. Agent behavior and logs ◦ Check the LiveKit agent/gateway logs around the time of 486/flood. Look for: ▪︎ Job/dispatch rejections, exceptions, or crash traces. ▪︎ Lines showing "reject" or "busy" states. ◦ Confirm whether the agent process is still running and healthy. 1. SIP traces / packet capture ◦ Capture a short pcap (tcpdump) on the LiveKit VM around the event and inspect for: ▪︎ Multiple INVITEs from the call center in quick succession. ▪︎ Retransmissions or duplicate messages that could look like a flood. 1. Metrics / VM resource usage ◦ Monitor CPU, memory, and network usage at the time of the issue to spot saturation. ◦ Check process counts and thread usage of the SIP gateway/agent. 1. Dispatch / configuration settings ◦ Check agent dispatch settings: are rejected jobs preventing further dispatching? Is the dispatch pool large enough? ◦ Inspect anti-flood/DoS thresholds on your SIP gateway. See if thresholds are too low and classify normal traffic as a flood. 1. Call center behavior (quick checks) ◦ Confirm the call center is not unintentionally retrying calls rapidly on failure (which would create a burst). ◦ Verify they are routing to the correct SIP endpoint and not sending duplicate calls from multiple upstream nodes. How to fix (concrete actions) • If LiveKit is actually overloaded: ◦ Scale agent instances (add more agents or gateway workers). ◦ Increase VM resources (CPU/RAM) or distribute load across additional VMs. • If the SIP gateway is rejecting due to anti-flood: ◦ Increase anti-flood thresholds or whitelist the call center IP(s). ◦ Apply more permissive rate limits while you tune production limits. • If network issues cause retransmissions: ◦ Fix packet loss or high latency on the path between the call center and LiveKit. ◦ Verify NAT/firewall behavior and disable or adjust SIP ALGs if present. • If an agent or job-handling bug: ◦ Restart the agent/gateway to restore service while you investigate logs. ◦ Fix the agent logic so it returns to ready state after calls complete or on transient errors. Next steps I can help with • If you share the following pieces I can give exact next commands and targeted config changes: ◦ LiveKit server and SIP gateway/agent version. ◦ A short excerpt of the SIP/gateway log around the 486/flood. ◦ A brief summary of call rate (calls per second or concurrent calls) when the issue happens. ◦ VM CPU/RAM metrics at the time (or current resource sizes). If you prefer, tell me whether you want to start with LiveKit logs or a network pcap and I’ll provide exact commands to collect them and what to look for. Sources: Why do jobs stop after agent rejection? | LiveKit | Agents telephony integration | LiveKit Documentation