{“level”: “WARNING”, “name”: “livekit.agents”, “message”: “process memory usage is high”, “memory_usage_mb”: 553.36328125, “memory_warn_mb”: 500, “memory_limit_mb”: 0, “pid”: 104, “job_id”: “AJ_76SpRUKaiYeK”, “room_id”: “RM_JJ2RJuopMbFh”, “timestamp”: “2025-12-16T105610.676310+00:00"} What’s issue

Thanks for reaching out, Madhav. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

it’s realtime llm agent

Thanks — knowing it's a realtime LLM agent helps. What this warning means for a realtime LLM agent • The agent exceeded your warning threshold (553.36 MB vs 500 MB). It’s a warning only because

memory_limit_mb

is 0, so the process won’t be auto-killed. • Realtime LLM agents commonly spike memory when loading models, keeping model contexts/token caches, or handling multiple simultaneous sessions. Practical next steps 1. Quick checks ◦ Is this recurring or a single spike? ◦ Which model size are you running (tiny / base / larger)? ◦ How many concurrent sessions or in-flight requests per agent? 1. Immediate mitigations ◦ Set a reasonable

memory_limit_mb

to prevent runaway memory (pick a limit slightly above typical peak after testing). ◦ Increase the warning threshold if this usage is expected and safe for your host memory. 1. Reduce per-agent memory ◦ Use a smaller model or a distilled/quantized variant. ◦ Limit concurrency per agent so each worker handles fewer sessions. ◦ Reduce retained context length or aggressively prune token caches between calls. 1. Architecture/scale options ◦ Deploy additional agent workers to spread sessions across processes. ◦ Offload heavy LLM work to separate services (inference cluster) and keep the realtime agent as a thin orchestrator. If you want, tell me: • model name/size, typical concurrency, and whether this is repeated — I’ll recommend a concrete

memory_limit_mb

and concurrency setting.

I am using google realtime mode and this is recurring

Thanks — since you’re running a Google realtime LLM agent and the high memory usage is recurring, here’s a targeted plan to diagnose and reduce it. Immediate checks (tell me the answers if you can) • Which Google realtime model/size are you using? • Typical concurrent sessions per agent and average session length (messages/tokens). • Any custom context retention or caching you’ve enabled. • Host machine RAM and whether multiple agents run on the same host. Short-term mitigations • Set

memory_limit_mb

to a value slightly above your observed peaks (e.g., if peaks ~550 MB, try 700–800 MB) so a runaway process is constrained. • Reduce concurrency per agent so each process handles fewer simultaneous sessions. • Trim context length or drop inactive session contexts sooner to free token caches. Medium-term changes • Use smaller/quantized model variants if latency and quality allow. • Move heavy inference to a dedicated inference pool (separate processes/hosts) and keep realtime agents as lightweight proxies. • Add horizontal capacity (more agent workers) so load per agent falls. Monitoring and tuning • Track memory over time per PID/job to identify growth patterns (steady leak vs spikes). • If growth is steady between requests, suspect retained caches or a memory leak; if spikes align with model loads or bursts, it’s capacity/concurrency. If you share model size, concurrency, and whether memory grows steadily or in spikes, I’ll recommend a concrete

memory_limit_mb

and concurrency settings.

gemini-2.5-flash-native-audio-preview-09-2025

Thanks — since you’re using the Gemini 2.5 realtime audio model and this is recurring, I can recommend concrete limits and changes, but need a few quick details: 1. Typical concurrent sessions per agent (average and peak)? 2. Average session length in tokens or messages, and whether you retain full context between turns. 3. Host RAM and whether multiple agent processes run on the same host. 4. Any current memory-related settings (

memory_limit_mb

and warning threshold). With those I’ll suggest a

memory_limit_mb

, a recommended concurrency cap, and immediate config changes to reduce recurrence.

10 concurrent session, avg session 45 mins yes nop no current settings

Thanks — with Gemini 2.5 realtime audio, 10 concurrent sessions and ~45 min sessions, here are concrete, practical recommendations. Recommended limits • Set memory_limit_mb = 900 (caps process before it destabilizes). • Set memory_warn_mb = 700 (earlier alert so you can react before hitting the limit). Concurrency and agent sizing • Limit each agent to 4–6 concurrent sessions. At 10 sessions total, run 2–3 agents per host so no single process handles all sessions. • If you keep 3 agents, set per-agent concurrency to 4 (3 agents * 4 = 12 capacity). Configuration changes • Configure agents to drop or truncate inactive session context after ~5–10 minutes of inactivity to free memory. • If possible, avoid keeping full conversation history in memory; store long-term context externally and fetch on demand. Operational actions 1. Deploy the new limits and per-agent concurrency caps. 2. Monitor memory per PID for 24–72 hours to confirm peaks stay below 900 MB and warnings fire near 700 MB. 3. If processes still spike above limit, reduce per-agent concurrency further or add more agents/hosts. Longer-term options • Use a quantized/smaller model variant if latency/quality tradeoffs allow. • Offload heavy model work to a dedicated inference pool and keep realtime agents as thin proxies. If you want, I can: • Provide the exact config snippet to set

memory_limit_mb

and

memory_warn_mb

for your agent setup, or • Recommend monitoring metrics/alerts to add while you test these limits. Which would you like?

yes please provide both

Below are the config snippets and monitoring recommendations for your Gemini 2.5 realtime audio agent (10 concurrent sessions, ~45 min avg). I’ll provide an example agent config, an example systemd/docker placement suggestion, and a short monitoring/alerting checklist. 1) Agent config snippet (set memory warn/limit and concurrency) • Adjust keys to match your agent config format; replace placeholders as needed.

agent:
  job_id: AJ_76SpRUKaiYeK
  room_id: RM_JJ2RJuopMbFh
  realtime_llm:
    model: "gemini-2.5-flash-native-audio-preview-09-2025"
  memory:
    memory_warn_mb: 700
    memory_limit_mb: 900
  concurrency:
    max_sessions_per_agent: 4
  context:
    inactivity_drop_seconds: 600   # drop/truncate context after 10 minutes idle
    max_retained_tokens: 2048

2) Process placement / scaling suggestion • Run 3 agent processes (or hosts) to cover 10 sessions: 3 agents * 4 concurrency = 12 capacity. • If using containers, set container memory limit slightly above

memory_limit_mb

(e.g., 1 GiB) to align host-level enforcement. Docker run example (adjust names/IDs):

docker run --name livekit-agent-1 --memory=1g \
  -e AGENT_CONFIG=/config/agent1.yaml \
  livekit/agent:latest

3) Monitoring & alerting checklist • Metrics to collect per process (PID/job): ◦ RSS memory (MB) ◦ CPU % ◦ Sessions count / active connections ◦ Average latency per request • Alert rules: ◦ Warn: memory > 700 MB for 2 consecutive minutes (send Slack/email) ◦ Critical: memory > 900 MB for 30s (auto-restart or remove from rotation) ◦ Sessions > configured concurrency (trigger scale-out) • Logs to watch: ◦ OOM or container kill events ◦ Realtime LLM errors/timeouts and model load events • Visualization: ◦ Dashboard with per-agent memory timelines, concurrency, and request latency (24–72h view) 4) Operational runbook (quick) 1. Deploy config changes to one agent. 2. Observe for 24h: memory peaks should stay <900 MB and warnings near 700 MB. 3. If still >900 MB, lower per-agent concurrency to 3 or add another agent host. 4. If memory grows steadily between requests, capture heap/profile to inspect retained caches (may indicate leak or large retained contexts). If you want, I can produce: • a ready-to-deploy agent YAML for your environment, or • example Prometheus alert rules and a Grafana dashboard JSON for the monitoring checks. Which would you like?