Hey team — hitting a reliable failure on Wi-Fi net...
# agents
c
Hey team — hitting a reliable failure on Wi-Fi network switches with livekit-agents (Python). Two failure modes from the same trigger: Stuck-in-reconnecting — connection_state_changed fires with state 2, never returns to 1. A TCP probe to the LK host stays reachable (30–100ms RTT) throughout the 30+ seconds the SDK is wedged, so it's not the network — the reconnect logic is stuck at the application layer. Ghost-room — connection_state_changed fires with state 1 (claims reconnected) but remote_participants is the stale pre-blip snapshot — no audio frames flow and participant_disconnected never fires for the participant that's actually gone. User-visible outage on a brief network blip is ~60s. We're currently working around it by monitoring connection_state_changed and force-evicting via RoomService.RemoveParticipant when the SDK doesn't recover, but would really appreciate guidance on what the SDK is supposed to do on the recovery path, or whether a fix is planned. Happy to share logs / repro steps if helpful.
t
Hi Matan Porat, ⚠️ This channel will be deleted soon ⚠️ Please use our new community at https://community.livekit.io for technical discussions and to post technical questions. The Slack community is not going away, but we are migrating technical questions to our new forum. Please see full details here.
p
This looks like a LiveKit SDK recovery/state sync issue rather than a network problem, especially since RTT stays stable and the room state desyncs after reconnect. Your workaround makes sense, but I’d also suggest adding a forced full room resubscribe/rejoin on state 2 → timeout, because the “ghost room” usually happens when the client doesn’t fully resync participant state after reconnection. If you can share logs/repro, I can help you narrow whether it’s SDK-side or lifecycle handling and suggest a more stable recovery flow. @cuddly-microphone-78836
c
Environment - livekit-agents==1.4.1 (also reproduced on 1.5.6) - livekit==1.0.25 (Python wrapper around the Rust livekit-ffi) - livekit-api==1.1.0 - Python 3.14 - macOS (local dev) and LiveKit Cloud (production) - RUST_LOG=livekit=debug,livekit_api=debug --- Repro 1. Start an AgentSession worker, have a remote participant join the room and exchange audio normally. 2. Cause a brief network blip on the agent host — Wi‑Fi network switch on macOS works every time; production hits it on natural network blips. 3. Observe: WS keepalive ping times out, SDK enters Resume strategy, then stalls. Two observed end-states: - Variant A (stuck-in-reconnecting): connection_state_changed → 2 (Reconnecting), never returns to 1. No further SDK activity for ~60s. - Variant B (ghost-room): connection_state_changed → 1 (Connected) but remote_participants is the stale pre‑blip snapshot. No audio frames flow and participant_disconnected never fires. Same SDK-side stall, different surface symptom. (Related: livekit/agents#1581.) --- Smoking-gun logs The wedge (Variant A, the 2s window before we killed the process): 182443.187 WARN livekit::rtc_engine - received session close: "signal client closed: \"ping timeout\"" UnknownReason Resume 182443.188 ERROR livekit::rtc_engine - resuming connection... attempt: 0 <silence — no ICE candidates, no further signaling, no errors, no attempt: 1> Earlier run where we let the SDK self-recover (same trigger, same attempt: 0 entry, no kill switch): 151555.557 ERROR livekit::rtc_engine - resuming connection... attempt: 0 <silence for ~65s> 151700.319 ERROR livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally 151700.319 ERROR livekit::rtc_engine - restarting connection... attempt: 1 → Resume does eventually escalate, but only when the next ping cycle independently times out ~65s later. That's the user-facing outage. close event never fires during the wedge. @session.on("close") handler does not run. await session.aclose() and await room.disconnect() both never return — confirmed with asyncio.wait_for(..., timeout=2.0) always tripping the timeout in this state. --- Why this looks like a bug, not expected behavior (LiveKit support confirmed this off-thread, paraphrasing their reply:) ▎ A ping timeout followed by resuming connection... attempt: 0 with no ICE restart and no escalation for ~60s ▎ is not consistent with the documented reconnect flow. Given that close never fires, this is a stalled resume ▎ path rather than expected behavior.
p
From what you’ve shared, it does look like the SDK is getting stuck in the Resume state without properly re-triggering ICE restart or escalating the reconnect attempt, which explains both the “stuck-in-reconnecting” and “ghost-room” outcomes. The fact that close never fires and aclose()/disconnect() hang also strongly points to a stalled internal resume loop rather than expected recovery behavior. At this stage, I’d treat it as a low-level LiveKit Rust FFI / resume-state handling bug rather than an application-layer issue, especially since it’s reproducible across versions and environments. If you want, I can help you tighten your current mitigation (so you don’t rely on timeouts alone) while this is escalated upstream, and suggest a safer recovery pattern for production. Let's connect
@cuddly-microphone-78836