steep-balloon-41261
03/25/2026, 8:22 AMtall-airline-16756
03/25/2026, 8:22 AMtall-airline-16756
03/25/2026, 8:23 AMrecord=False) to avoid using minutes.
Possible reasons for the discrepancy
• A dashboard display or metering bug showing incorrect percentage.
• Some recordings counted by backend telemetry that aren’t visible in the table you looked at (partial/failed sessions or short segments).
• You may be looking at a different time window, region, or a cached/older view of usage.
Recommended next steps (quick checks you can do now)
1. Verify you’re viewing the same project and the current month in the Cloud dashboard (ensure no date filters or cached view).
2. Check project-level Data & Privacy settings to confirm recording is enabled/disabled and whether any agents set record=True by default.
3. Review detailed billing/usage report in the project dashboard for recorded minutes and any breakdown by session or agent.
4. If you have multiple similar projects, confirm the email referenced the exact project name in the header (email said satara-diagnostic-dev — double-check that matches the project you inspected).
What I can help with next
• If you’d like, tell me:
◦ The exact project ID (if different from satara-diagnostic-dev),
◦ The month/date range you’re looking at,
◦ Whether you’ve enabled recording at project or agent level.
I’ll use that to help pinpoint whether this is a dashboard display issue, an actual usage spike, or a config causing extra recordings.
Sources: Pricing | LiveKit | Pricing | LiveKit | Towards a future-aligned pricing model | LiveKit | Agents on LiveKit Cloud | Deploy voice AI on global infrastructure | LiveKi... | LiveKit Agent Observability | Debug and optimize voice agents | LiveKit, with interactive elementsblue-school-51670
03/25/2026, 12:02 PMtall-airline-16756
03/25/2026, 12:02 PMlk agent deploy, the dashboard shows one version as "current" and the other as "draining", but new connections are still landing on the old version. Here’s what’s happening and what you can do.
What “draining” means
• When you deploy, LiveKit performs a rolling update: the new version becomes current and the old version switches to draining.
• The draining version stops accepting new sessions but continues serving any active sessions until they end. This ensures no one is abruptly disconnected.
Why you still see connections to the old version
• If existing sessions that began on the old version are still active, those connections remain on the draining instance. New sessions should use the new current version immediately.
• If new connections are still going to the draining version, it likely means those sessions were initiated before the deploy or your client is reconnecting to an existing session tied to the old version.
How long until the old version is fully closed
• There is no fixed timeout. The draining version stays active until all its active sessions finish naturally.
• Therefore the time-to-close equals the remaining duration of those sessions.
How to check and speed up the transition
1. Monitor active sessions and version status:
◦ Run lk agent status to see which sessions are active on each version.
◦ Check lk agent logs for session lifecycle events.
1. Let sessions finish naturally whenever possible — this is the safest way to preserve UX.
2. If you need a faster cutover and can tolerate forced disconnects:
◦ Manually end the active sessions (from your app or dashboard) so the draining version can terminate sooner.
◦ Note: abruptly terminating sessions may interrupt users and their recordings/transcripts.
1. Confirm clients create new sessions after deploy:
◦ Ensure clients request fresh sessions (not reusing old session IDs or reconnecting to legacy session endpoints).
If you want, tell me:
• The output of lk agent status (or how many active sessions the draining version shows), and
• Whether you can allow existing sessions to finish or need an immediate cutover.
I can walk you through the exact commands to inspect sessions and safely expedite the transition.
Sources: Agent deployment overview | LiveKit Documentation | Deploy and scale agents on LiveKit Cloud | LiveKit | examples/drive-thru | livekit/livekit_composite | DeepWiki | Server options | LiveKit Documentation, with interactive elementsblue-school-51670
03/25/2026, 12:08 PMtall-airline-16756
03/25/2026, 12:08 PMlk agent status output and your question about why connections still go to the old version.
What your lk agent status shows
• You’re using project satara-diagnostic-dev and agent CA_eNqsqE5cjkfn.
• There is one active version listed: v20260325115359 in ap-south, Status = Running, Replicas 1/1/8, deployed at 2026-03-25T115543Z.
What this means
• The status output shows the currently running version and its resource usage. It does not explicitly list a separate “draining” version in this output — only the running/current one is shown.
• If you still see clients connecting to an older instance after deploy, it’s likely because those clients are reconnecting to existing sessions that were created on the older version (or your client is reusing session IDs or connection tokens tied to the old session). The deployed agent will perform a graceful handoff: it prevents new sessions on the old instance while allowing in-progress sessions to finish.
How to confirm if any old/draining version still has active sessions
1. Run lk agent list (if available) to show all versions and their states (current vs draining).
2. Use lk agent status on any other agent IDs you may have — sometimes older replicas or agents appear under different IDs.
3. Check active sessions and which version they’re attached to:
◦ Run lk agent status and look for session counts in the detailed view or use any session-inspection command your deployment provides.
◦ Check logs with lk agent logs to see session creation/reconnect events and which version handled them.
Why new sessions should go to the new version
• New sessions created after deployment are routed to the new current version immediately. If a client is still routed to the old version, it’s because the client is reconnecting to an existing session (not creating a fresh session).
If you want to force cutover faster (with caution)
• Allow existing sessions to end naturally (recommended).
• Or manually terminate active sessions that are keeping the draining version alive (this will disconnect users and may interrupt recordings).
Next steps I can help with
• If you paste the output of lk agent list (or the full lk agent status if there’s a more detailed form), I’ll check for any draining version entries.
• If you can share whether your clients reuse session IDs/tokens on reconnect, I’ll suggest how to force new session creation from the client side so they connect to the current version.
Tell me which option you prefer and I’ll walk you through the exact commands and checks.
, with interactive elementsblue-school-51670
03/25/2026, 12:11 PMtall-airline-16756
03/25/2026, 12:12 PMlk agent list output and your note that calls arrive via SIP (so there shouldn't be existing sessions to keep the old version alive).
Quick summary of what I see
• Your lk agent list shows a single agent version: v20260325115359 deployed at 2026-03-25T115543Z.
• You said callers arrive via SIP from phone numbers (not reusing old session IDs), so existing agent sessions are unlikely to be the reason old-version routing persists.
What’s likely happening
• After deploy LiveKit performs a rolling update; the old instance may be in a draining state even if it’s not listed separately by lk agent list. New sessions are supposed to be routed to the new/current version immediately.
• For SIP calls there are a few common reasons new calls still reach the old runtime even though sessions wouldn’t be reused:
◦ SIP routing or trunk provider caching (DNS, SIP route, or upstream carrier caching) causing calls to keep hitting the old endpoint for a short time.
◦ SIP proxy configuration or SIP registration still pointing to the old endpoint/IP.
◦ An external SIP gateway or load balancer with sticky routes that haven’t refreshed.
◦ Short DNS TTL or SIP proxy caching delays — these can keep traffic going to the previous endpoint for a few minutes to hours depending on TTLs/caches.
Checks and steps to diagnose
1. Confirm the active endpoint the call is arriving at:
◦ Inspect your SIP gateway or carrier logs to see which IP/hostname they are dialing.
◦ Check LiveKit agent logs at deploy time for inbound SIP call handoff (use lk agent logs) and note which version handled the call.
1. Verify your SIP trunk / SBC / proxy config:
◦ Ensure the trunk target was updated (or that it uses a stable hostname that resolves to the new version).
◦ Check for any sticky routing or registration binding to the old instance.
1. Check DNS and cache behavior:
◦ If you use a DNS name for the agent endpoint, confirm its TTL and whether your carrier respects TTLs; some carriers cache aggressively.
1. Reproduce a test call while tailing logs on the agent:
◦ Run lk agent logs --follow (or equivalent) while placing a test SIP call to confirm which version receives the INVITE.
1. If you need an immediate cutover:
◦ Restart or deregister the old instance from your SIP routing (if possible) so the SIP provider will attempt the updated route — be careful: this will drop any active calls that might be on that instance.
◦ Ask your SIP provider to flush caches (if they offer that).
If you’d like, I can help next with any of these:
• Interpret relevant lines from lk agent logs if you paste a test-call trace.
• Show exact commands to tail logs while you place a test call.
• Walk through what to check in your SIP trunk/SBC to ensure it points to the current runtime.
Tell me which you prefer and paste a short log snippet from a recent incoming call (or run a live test and paste the logs). I’ll walk you through the next steps based on what the logs show.
, with interactive elementsblue-school-51670
03/25/2026, 12:15 PMtall-airline-16756
03/25/2026, 12:15 PMregistered_worker on the new version but no incoming-call logs when you place a call, and that indicates the calls are still reaching the old (draining) runtime. I’ll explain why that can happen and give focused troubleshooting and safe remediation steps.
Why this happens (short)
• Deploys use a graceful rolling update: the old runtime is put into draining mode and should stop accepting new sessions, but in practice SIP traffic can continue to hit the old instance for reasons outside LiveKit (carrier/SBC/TLS session caching, SIP proxy sticky routing, or registration bindings).
• Because SIP call setup can be cached or routed upstream, an INVITE may still reach the old runtime even though the agent name/ID now points to the new version.
Targeted checks to confirm where calls land
1. Tail logs on both versions while placing a test call:
◦ On the new version: run lk agent logs --follow and watch for INVITE or call-handling lines.
◦ If you can access logs for the older instance, tail those too. If only the new version shows logs up to registration, but the old instance shows the call activity, that proves traffic is still routed to the old runtime.
1. Inspect SIP carrier / SBC / proxy logs for the incoming INVITE:
◦ Confirm the destination IP/hostname the carrier dials and the timestamp — this shows which backend handled the call.
1. Verify whether your SIP trunk uses persistent registration or a static binding:
◦ Carriers/SBCs sometimes maintain persistent sessions or reuse previous targets for a period after a change.
Actions to accelerate cutover (ordered from safest to fastest)
1. Recommended (safe): Wait for in-progress calls/connections on the old runtime to finish.
◦ This keeps user sessions intact and is what draining is designed to do.
1. Low-impact: Force new fresh sessions from callers (if possible).
◦ Ask a test caller to hang up and re-dial (not a solution for all callers but helpful for tests).
1. Moderate (if acceptable): Ask your SIP provider/SBC to flush caches or reload routing for the agent name.
◦ Some providers can purge cached routes or force re-resolution of DNS/targets.
1. Faster / disruptive: Deregister or restart the old instance so it stops answering.
◦ This will drop any active calls on the old version and force the carrier to route to the new runtime. Only do this if you can tolerate dropped calls.
◦ How to do this: stop the old agent process or remove it from your SIP routing (whatever method your infra supports). After that, verify with a test call and lk agent logs --follow.
Diagnostic commands and what I’d like you to run next
1. While you place a test SIP call, run:
◦ lk agent logs --follow
◦ Paste any new lines you see (or confirm that nothing new appears).
1. If you can get logs for the old instance, tail them at the same time and paste the call-related lines.
2. From your SIP provider/SBC, capture the incoming INVITE log line and share the target IP/hostname and timestamp.
If you prefer, I can walk you through:
• Exact commands to tail logs and grep for SIP-specific entries.
• How to identify the routing target in your carrier logs.
• The safest way to stop the old instance in your environment.
Tell me which diagnostic step you want to run now (tail new logs, tail old logs, or fetch carrier/SBC trace), and I’ll give the precise commands and what to look for.
, with interactive elements