does livekit agent support graceful shutdown?

Thanks for reaching out, Jura. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i run it on kube with this setting, with ongoing call, it does not for the call to end

# Timeout of progress when update
progressDeadlineSeconds: 300

# Strategy to update deployment
strategy:
  # type: Recreate
  rollingUpdate:
    maxUnavailable: 0
    maxSurge: 1

# Timeout to force terminate pod
terminationGracePeriodSeconds: 300

lifecycle:
  preStop:
    exec:
      command:
        - sh
        - "-c"
        - sleep 15

Thanks — I understand you’re running the agent on Kubernetes and ongoing calls aren’t finishing during pod termination. A few targeted questions and checks will help narrow this down. 1) Which agent mode are you running? (

start

= production drain behavior;

dev

= no drain) 2) Which agent runtime/language are you using (Python, Go, or the reference agent)? 3) What value is the agent’s

drain_timeout

set to (default 1800s / 30m)? Are you passing it via server constructor or CLI (

--drain-timeout

)? 4) Are you using a wrapper process or an init script for the container command (any wrapper can mask SIGTERM)? If so, are you using

exec

to replace the shell with the agent process? Checks and suggestions to try now: • Ensure the agent runs in

start

mode (not

dev

). Only

start

drains active jobs. • Increase Kubernetes pod termination time to exceed the agent

drain_timeout

(or increase

drain_timeout

to be ≤ pod grace period). For example, set

terminationGracePeriodSeconds

>

drain_timeout

. • Replace the preStop sleep with a no-op or ensure it doesn’t prevent SIGTERM reaching the agent; prefer letting Kubernetes signal the process directly. If you must use a script, use

exec

so the agent process receives signals. • Confirm your Deployment strategy allows draining: rolling update with

maxUnavailable: 0

and

maxSurge: 1

is fine, but the pod’s grace period must allow the full drain window. • Check agent logs on shutdown for “draining worker” or asyncio timeout messages — that shows the drain flow and whether the timeout fired. If you share the agent mode, runtime, current

drain_timeout

value, and whether you use a wrapper script, I’ll give exact configuration changes and the minimal pod spec snippets to ensure calls can finish. , with interactive elements

1. CMD ["python", "main.py", "start", "--log-level=debug"] 2. python 3. default, didn't config anything 4. this docker cmd

CMD ["python", "main.py", "start", "--log-level=debug"]

Thanks — helpful details. You're running the Python agent in

start

mode (good) and using the default

drain_timeout

(1800s / 30m). The likely problem: your Kubernetes pod is being terminated before the agent's drain window completes or the SIGTERM isn't reaching the agent process. What to change (minimal, ordered): 1. Increase Pod grace period to at least the agent drain timeout ◦ Set

terminationGracePeriodSeconds

> 1800 (e.g., 1900) or set to a value you want and then set the agent

drain_timeout

to be <= that value. 1. Configure agent

drain_timeout

explicitly to match your pod grace period ◦ Start the agent with

--drain-timeout <seconds>

(e.g., 1800) or set in

AgentServer(drain_timeout=...)

so Kubernetes and agent timeouts align. 1. Remove or avoid preStop sleep that delays SIGTERM delivery ◦ Your current

preStop: sleep 15

delays termination sequencing but is unnecessary; it can also hide timing issues. If you keep a preStop hook, ensure it uses

exec

to forward signals (or better, remove it). 1. Ensure the container process receives signals directly ◦ Your Docker CMD runs

python

directly, which is fine. Just avoid wrapping with additional shell processes that do not

exec

the agent; wrappers can block SIGTERM from reaching Python. 1. Verify behavior with logs ◦ On termination, agent logs should show “draining worker” and then either job completion or an asyncio timeout. If you see no “draining worker”, SIGTERM may not be reaching the process. Recommended quick example: • Set pod spec: terminationGracePeriodSeconds: 1900 • Run agent: CMD ["python", "main.py", "start", "--log-level=debug", "--drain-timeout=1800"] If you want, I can produce the exact Deployment pod spec snippet with these settings and the logs/commands to validate the drain sequence. Which drain timeout would you prefer (keep 1800s or shorten/lengthen)? , with interactive elements