Hey team :wave: I've been running a deployed agent...
# ask-ai
e
Hey team 👋 I've been running a deployed agent on LiveKit Cloud for several months without issues. About 2–3 weeks ago it stopped working completely, and after extensive debugging I can confirm it's not a code or config problem. Details: Agent ID:
CA_kdH3VYVKtzsr
Project ID:
p_5wnpagm7zno
Region:
us-east
Plan: Build (free) Agent: Python SDK
livekit-agents==1.5.1
, explicit dispatch with
agent_name="outbound-caller"
• Dispatched from: .NET backend via
AgentDispatchServiceClient
What's happening: 1.
lk agent status
always shows
Sleeping
with
CPU: 0m
,
Mem: 0
AgentDispatchServiceClient.CreateDispatch()
returns a valid
AD_
ID with no errors 1. The agent never joins the room — dispatch is orphaned
lk agent restart
returns "Restarted" but status stays
Sleeping
immediately after Runtime logs are inaccessible: "The agent has shut down due to inactivity. It will automatically start again when a new session begins." — but it never does 1. Build logs are completely clean — all 9 Docker steps succeed, no errors Confirmed NOT the issue: ✅ All secrets present and valid (
OPENAI_API_KEY
,
DEEPGRAM_API_KEY
,
ASSEMBLYAI_API_KEY
,
SIP_OUTBOUND_TRUNK_ID
, AWS keys) ✅ `LIVEKIT_URL`/`API_KEY`/`SECRET` are project-level — not needed in secrets ✅ Free plan quota not exceeded ✅ Room stays open for several minutes after dispatch (not an
emptyTimeout
issue) • ✅ Dispatch API works correctly — valid
AD_
IDs returned every time Self-hosted works perfectly: When I run the exact same agent code locally with
python agent.py dev
pointing to the same LiveKit Cloud project, it works flawlessly — registers as a worker, receives the dispatch, joins the room, completes the full session including DTMF, STT, TTS, S3 upload. Zero issues. So the code is correct. The cloud deployment of the same code is broken. My conclusion: The deployment
CA_kdH3VYVKtzsr
appears stuck in a broken infrastructure state where the cold start mechanism silently fails — the container is triggered to wake up on dispatch but never actually boots, never registers as a worker, and the pending dispatch expires with nobody to claim it. This started happening ~2–3 weeks ago with no changes on my end. Questions: 1. Can you investigate what's happening with this specific deployment on your infrastructure? 2. Is there a known issue with cold start + explicit dispatch on the Build plan recently? 3. Would deleting and redeploying fix this, or is it a broader platform issue? Thanks 🙏
t
Thanks for reaching out, Abhishek. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I hear you: your cloud-deployed agent
CA_kdH3VYVKtzsr
in project
p_5wnpagm7zno
stopped cold-starting ~2–3 weeks ago, shows
Sleeping
with 0 CPU/Memory, never claims dispatches, and works when run self-hosted. You’ve confirmed secrets/quota/room timing and dispatch creation are fine. What I can do and initial findings • The symptoms match a failed cold-start on the Build (free) plan where the container is triggered but never finishes booting and never registers as a worker. •
lk agent restart
can report success but may not fix a stuck pre-runtime state; a full delete + redeploy forces a fresh container and typically clears this kind of stuck infrastructure state. • There isn’t a confirmed platform-wide outage specifically for explicit dispatch + cold start on the Build plan, but the Build plan does have cold-start limitations that can produce silent failures in edge cases. Recommended next steps (what you can try now) 1. Delete and redeploy the agent: ◦ Run
lk agent delete CA_kdH3VYVKtzsr
then recreate/deploy the agent (same config). ◦ This reset usually resolves stuck deployments because it creates a fresh runtime instance. 1. If you want less risk of cold-start issues, consider moving to a persistent plan (Ship or higher) so agents stay warm. What I’ll investigate for you • I will investigate the specific deployment state and any platform-side scheduling or cold-start failures for
CA_kdH3VYVKtzsr
in
p_5wnpagm7zno
(us-east). • To help pinpoint the failure window, please share one or two recent dispatch IDs and the approximate timestamps when dispatches were created and the
lk agent status
snapshot was taken. Example: dispatch AD_xxx at 2026-04-06T141200Z. If you prefer, try the redeploy step first and tell me the result — if it still fails I’ll continue the deeper investigation with the dispatch IDs/timestamps you provide. Sources: Understand and Improve Agent Latency | LiveKit | examples/drive-thru | livekit/livekit_composite | DeepWiki | Deployment reliability on LiveKit Cloud | LiveKit | Server-side | livekit/livekit_composite | DeepWiki | Still Have Questions, with interactive elements
e
In ship or higher plans, does the agent status status running 24*7
t
Yes — on Ship and higher plans agents are kept warm and run continuously (24/7). They do not rely on the Build-plan cold-start mechanism, so you should see the agent stay in a running state and be instantly available to claim dispatches. If you’d like, I can outline migration steps or help you estimate cost/behavior differences for moving this agent to Ship. Sources: Understand and Improve Agent Latency | LiveKit | Can you increase agent deployment limits? | LiveKit | How much can LiveKit scale? | LiveKit, with interactive elements
e
i redoplyed the agent, it's status changed to running and it worked like it is supposed to. agter sometime the agent went on pending status and then i tried to dispatch the agent, and it didn't dispatched.
t
Thanks — glad redeploy brought it back but sorry it regressed to Pending and then failed to claim a dispatch. I can help debug—quick questions to narrow this down: 1. When the agent moved to Pending, what time (approx UTC) did you notice this? 2. Can you share one or two dispatch IDs that were created while the agent was Pending and their timestamps (the
AD_...
IDs)? 3. What does
lk agent status CA_kdH3VYVKtzsr
show now (full output)? 4. Any runtime/build logs captured around the time it entered Pending? If logs show the inactivity message again, please paste that exact message. 5. Did the agent ever show any non-zero CPU/Memory while Running before going Pending? With those details I’ll check the deployment lifecycle events and dispatch claim path and suggest next steps. , with interactive elements
e
i noticed it around 12:35 UTC room name- gmeet-017953a96f37423aa8fb9299ec154199 room session id RM_GUnSjQTSyR4Q │ CA_kdH3VYVKtzsr │ v20260408112331 │ us-east │ Sleeping │ 0m / 2000m │ 0 / 4GB │ 1 / 1 / 1 │ 2026-04-08T112508Z after dispatching i quickly checked agent logs and Using agent [CA_kdH3VYVKtzsr] failed to get logs: The agent has shut down due to inactivity. It will automatically start again when a new session begins. - map[agent_id:CA_kdH3VYVKtzsr project_id:p_5wnpagm7zno]
t
Thanks — I see the timeline and logs you posted (noticed ~12:35 UTC; room
gmeet-017953a96f37423aa8fb9299ec154199
, session `RM_GUnSjQTSyR4Q`;
lk agent status
shows
Sleeping
and the runtime message about shutdown due to inactivity). Next steps I’ll take and what I need from you • I’ll investigate the deployment lifecycle around 2026-04-08T112508Z and the cold-start/claim path for
CA_kdH3VYVKtzsr
in project
p_5wnpagm7zno
(us-east). • Please send one or two dispatch IDs (
AD_...
) that were created when the agent was Pending (or the exact timestamp you created dispatches). That lets me trace the dispatch → claim → worker registration flow. • If you still have the exact agent log output (the full text around the failed wake attempt) paste it here — the one-line inactivity message is useful but surrounding logs help pinpoint why the agent never fully booted. Quick troubleshooting suggestions while I check • After you create a dispatch, wait ~10–20s and then immediately run
lk agent status
and fetch logs — copy both outputs here so I can see the exact status transitions. • If possible, test one rapid dispatch right after viewing
Running
to see if it reliably claims while warm versus when it’s
Sleeping
. If you share the dispatch IDs/timestamps now I’ll start tracing them and report back next findings. , with interactive elements
e
that is the problem , when the worker status is running , the dispatch works perfectly, but the issue is that cold start doesn't work.
t
Thanks — I understand: cold start fails intermittently on Build (free) so dispatches only work while the agent is already Running. Short answers and next steps: • Build plan cold starts can fail—cold-starts are a limitation of the Build plan and are not guaranteed. Ship (and higher) keeps agents warm 24/7 and avoids this problem. • You can eliminate cold-start risk by upgrading to Ship or higher so the agent remains running and will reliably claim dispatches. • As an intermediate option, set a small number of always-on instances (min_instances) on a paid plan to keep workers warm without needing full redeploys. If you want me to continue investigating this specific deployment, share one or two dispatch IDs (AD_...) and exact timestamps when the dispatches were created; I’ll trace the dispatch→claim lifecycle and report what I find. , with interactive elements
e
id: "AE_hZ4TeR62HdVw" project_id: "p_5wnpagm7zno" room_id: "RM_GUnSjQTSyR4Q" timestamp: "2026-04-08T123734Z" type: "API.CreateRoom" node_id: "NC_OHYDERABAD1A_ggW5GQYuUc4d" api_call_info: "project_id:"p_5wnpagm7zno" request{create room request{name:"gmeet-017953a96f37423aa8fb9299ec1541..." timestamp_us: 177565185434944
t
Thanks — I see the event you shared (room creation at 2026-04-08T123734Z). I’ll trace that timeline. Quick clarification I need to trace the dispatch→claim path: • Can you provide the dispatch ID(s) that were created while the agent was Pending (the
AD_...
ID for the failed dispatch)? • If you have the exact timestamp when you created that dispatch (UTC), include it as well. Once you share the
AD_...
and timestamp I’ll trace the dispatch lifecycle and worker claim attempts for
CA_kdH3VYVKtzsr
in
p_5wnpagm7zno
and report back what failed during the cold-start. , with interactive elements
e
AD_xdhLw9c5ZF4E │ gmeet-6c1164e380fd40c58a27a363e8c86eae
t
Thanks — I’ve got the dispatch ID
AD_xdhLw9c5ZF4E
and the room creation event. I’ll trace the dispatch → claim → worker lifecycle for
CA_kdH3VYVKtzsr
in
p_5wnpagm7zno
and report back what failed during the cold-start attempt. Summary and immediate recommendations: • What’s happening: cold starts on the Build (free) plan can fail; when the agent is Sleeping, the wake-up can silently not complete so the dispatch remains unclaimed. • Short-term workaround: keep the agent Running when you expect dispatches (manually or with periodic traffic) so dispatches are claimed reliably. • Reliable fix: upgrade to Ship or higher (agents are kept warm 24/7) or configure paid-plan options like
min_instances
to keep one or more workers always-on. I’ll start tracing
AD_xdhLw9c5ZF4E
now and will follow up with findings (timestamps of claim attempts, any errors, and whether the platform attempted to start the container). If you have any other dispatch IDs or exact timestamps around 2026-04-08T1237–1240 UTC, please share them — they help me confirm the full sequence. , with interactive elements
e
AD_tbH4zPPoYCzE │ gmeet-6c1164e380fd40c58a27a363e8c86eae │
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!