kind-postman-17041
05/05/2026, 4:26 PMtall-airline-16756
05/05/2026, 4:26 PMtall-airline-16756
05/05/2026, 4:27 PMlk agent status says "updating" even though calls are answered.
This is expected during a rolling deployment: the new agent version can serve traffic while old instances are still draining, so the UI/CLI can show "pending" or "updating" until all old instances finish. That’s why calls work but the status hasn’t settled.
Please try these quick checks:
• Confirm routing: are new sessions going to the new instances (look for recent session start logs tied to the new version)?
• Check agent logs for drain activity (search for "draining worker" or SIGTERM handling).
• Verify old-instance session counts drop to zero (per-version active sessions).
• Note timing: cloud rollout + default drain_timeout can take up to ~30–60 minutes to fully finish.
If you’d like, tell me:
• the agent version(s) shown in the UI/CLI,
• any recent deployment time,
• and any drain-related log lines you see—
and I’ll help interpret the logs and next steps.
Sources: Deployment reliability on LiveKit Cloud | LiveKit, with interactive elementskind-postman-17041
05/05/2026, 4:27 PMtall-airline-16756
05/05/2026, 4:28 PMkind-postman-17041
05/05/2026, 4:29 PMtall-airline-16756
05/05/2026, 4:29 PMkind-postman-17041
05/05/2026, 4:52 PMtall-airline-16756
05/05/2026, 4:52 PMlk agent status bouncing between "running" and "updating."
That oscillation is usually caused by startup/health-check timing, signal handling, or brief instance churn during rollout (even on first deploy). To pinpoint the root cause I need a few details:
• Agent version shown in the UI/CLI
• Deployment timestamp (when you first deployed)
• A short log excerpt around a transition (look for lines mentioning "draining", "SIGTERM", health-check failures, or restarts)
Send those and I’ll analyze the logs and recommend the exact next steps to stabilize the status.
, with interactive elementskind-postman-17041
05/05/2026, 4:55 PMtall-airline-16756
05/05/2026, 4:55 PMlk agent status still bounces between "running" and "updating." That typically means the control plane is seeing the instance start, fail a health check, or briefly restart — so we need runtime/health-check details (build logs won't show these).
Could you provide the following so I can diagnose quickly?
• Agent version shown in the UI/CLI
• Deployment timestamp (when you first deployed)
• A short excerpt of the agent runtime logs around a transition (look for lines mentioning health checks, draining, SIGTERM, restarts, or errors)
• Confirm whether the health-check endpoint responds with HTTP 200 and how long it takes to respond
If you paste the log excerpt here I’ll analyze it and recommend the concrete next steps to stop the oscillation.
, with interactive elementskind-postman-17041
05/05/2026, 5:00 PMtall-airline-16756
05/05/2026, 5:00 PMlk agent status CA_TiQxgTh9eux4
◦ Paste the full output.
1. Tail the agent runtime logs around a transition
◦ If you have CLI access where the agent runs, run:
▪︎ lk agent logs CA_TiQxgTh9eux4 --follow --lines 200
◦ If your environment uses kubectl, run:
▪︎ kubectl logs -l app=livekit-agent -c agent --since=10m --tail=200
◦ Paste any lines that mention drain, SIGTERM, health, error, or restart messages (even short snippets are helpful).
1. Check the health-check endpoint
◦ If you know the agent’s health URL, run:
▪︎ curl -i <HEALTH_URL>
◦ If unknown, tell me whether your agent is behind a load balancer or the cloud provides a health-check path and I’ll guide you to find it.
1. Note timing of a recent transition
◦ When you next see the status flip, capture the exact timestamp and include logs from ~1 minute before to 2 minutes after.
If you prefer, paste the outputs you can get now (even just the lk agent status output) and I’ll start analyzing.
, with interactive elementskind-postman-17041
05/05/2026, 5:03 PMtall-airline-16756
05/05/2026, 5:03 PM/health or /healthz. Please do the following:
1. Open the LiveKit Cloud Console and navigate to your agent (ID: CA_TiQxgTh9eux4). On the agent details page look for a Health or Endpoints section — the full health URL will be shown there (it will look like https://.../health or https://.../healthz).
2. Once you have the health URL, run:
◦ curl -i HEALTH_URL
Paste the full response here (status code and body). A healthy agent should return HTTP 200 quickly.
1. Grab runtime logs from the cloud Console:
◦ On the same agent details page there should be a Logs tab or a “View logs” link. Open it and filter for the last 30 minutes.
◦ Copy any lines around a status transition (search for drain, draining, SIGTERM, health, error, or restart). If you can’t find those terms, paste the last ~200 log lines.
1. If you can’t find the health URL or logs in the Console, tell me what you see on the agent details page (tabs/links available) and I’ll guide you step-by-step.
When you paste the health-check response and any log excerpts, I’ll analyze them and tell you the next concrete steps to stop the oscillation.
, with interactive elementskind-postman-17041
05/05/2026, 6:28 PMtall-airline-16756
05/05/2026, 6:28 PMdrain_timeout is 1800s by default). Because calls are still answered, the new instance is serving traffic correctly.
Next steps I recommend:
• Confirm a new instance is active (check the agent list for a different worker ID than CAW_yuKAKMycg8bS).
• Check the agent health endpoint (from the agent details page in the Cloud Console) returns HTTP 200 quickly.
• Monitor logs for any repeated drain/shutdown cycles or health-check failures that would cause continuous churn.
If you want, paste the agent status output (lk agent status CA_TiQxgTh9eux4) and the most recent 200 log lines from the Cloud Console and I’ll check for repeated cycles or health-check errors and suggest specific fixes.
, with interactive elementskind-postman-17041
05/05/2026, 6:30 PMtall-airline-16756
05/05/2026, 6:31 PMdrain_timeout is 1800s (30 minutes), so the UI/CLI can continue toggling until the draining instance fully exits or active sessions drop to zero.
Next, please check these two quick things and paste the results so I can confirm the cause and advise fixes:
1. Current worker list and statuses
◦ Run: lk agent status CA_TiQxgTh9eux4
◦ Paste the full output so I can see all worker IDs and their states.
1. Recent logs to detect repeated churn
◦ From the Cloud Console logs for this agent, paste the last ~200 lines (or any repeated blocks showing registered, draining, shutting down, health failures, or restarts).
If the agent list shows a new worker ID active and no repeated drain cycles, the status should stabilize after the remaining drain timeout. If you see repeated register→drain cycles in the logs or health-check failures, paste those lines and I’ll recommend targeted fixes (health-check tuning, investigate SIGTERM handling, or increasing stability during startup).
, with interactive elementstall-airline-16756
05/05/2026, 6:32 PMlk agent status CA_TiQxgTh9eux4.
1. Confirm health endpoint responds 200:
◦ From the agent details page in Cloud Console, get the health URL (likely /health or /healthz) and run:
▪︎ curl -i HEALTH_URL
◦ Paste the response.
1. Watch logs for repeated register→drain cycles:
◦ Paste the last ~200 log lines from the Cloud Console (look for repeated registered, draining, shutting down, health failures).
If the agent list shows a new worker active and you only see one drain cycle, the status should stabilize after the remaining drain timeout. If you see repeated register→drain cycles or health-check failures, paste those log lines and I’ll recommend targeted fixes (health-check tuning, investigate SIGTERM handling, or startup health latency).
, with interactive elementskind-postman-17041
05/05/2026, 6:33 PMtall-airline-16756
05/05/2026, 6:33 PM/health or /healthz) and run:
▪︎ curl -i HEALTH_URL
◦ Paste the response (status code and body).
If you prefer, paste the last ~200 log lines from the Cloud Console and I’ll scan them for the exact failure reason and recommend the specific fix (health-check tuning, restart, or redeploy).
, with interactive elementskind-postman-17041
05/05/2026, 6:34 PMtall-airline-16756
05/05/2026, 6:34 PM