proud-match-31207
11/13/2025, 3:19 PM{"message": "livekit::rtc_engine:453:livekit::rtc_engine - received session close: \"signal client closed: \\\"ping timeout\\\"\" UnknownReason Resume", "level": "WARNING", "name": "livekit", "pid": 77515, "job_id": "AJ_MhbAELG4WnuH", "timestamp": "2025-11-13T15:10:53.015125+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 0", "level": "ERROR", "name": "livekit", "pid": 77515, "job_id": "AJ_MhbAELG4WnuH", "timestamp": "2025-11-13T15:10:53.016054+00:00"}
cc: @refined-appointment-81829refined-appointment-81829
11/13/2025, 6:10 PMproud-match-31207
11/13/2025, 6:17 PMproud-match-31207
11/17/2025, 4:59 PM{
"message": "livekit::rtc_engine:453:livekit::rtc_engine - received session close: \"signal client closed: \\\"ping timeout\\\"\" UnknownReason Resume",
"level": "WARNING",
"name": "livekit",
"pid": 152786,
"job_id": "AJ_fyxVgrKTusfQ",
"timestamp": "2025-11-17T16:46:46.741406+00:00"
}
{
"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 0",
"level": "ERROR",
"name": "livekit",
"pid": 152786,
"job_id": "AJ_fyxVgrKTusfQ",
"timestamp": "2025-11-17T16:46:46.742088+00:00"
}
{"message": "error running user callback for local_track_published: room_handle: 4\nlocal_track_published {\n track_sid: \"TR_AMe7snshHNrUpa\"\n}\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/room.py\", line 560, in _listen_task\n self._on_room_event(event.room_event)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/room.py\", line 607, in _on_room_event\n lpublication = self.local_participant.track_publications[sid]\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^\nKeyError: 'TR_AMe7snshHNrUpa'", "level": "ERROR", "name": "root", "pid": 152786, "job_id": "AJ_fyxVgrKTusfQ", "timestamp": "2025-11-17T16:47:11.333984+00:00"}
Nearby Logs
Nov 17 22:17:11.336
{"message": "error running user callback for local_track_subscribed: room_handle: 4\nlocal_track_subscribed {\n track_sid: \"TR_AMe7snshHNrUpa\"\n}\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/room.py\", line 560, in _listen_task\n self._on_room_event(event.room_event)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/room.py\", line 616, in _on_room_event\n lpublication = self.local_participant.track_publications[sid]\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^\nKeyError: 'TR_AMe7snshHNrUpa'", "level": "ERROR", "name": "root", "pid": 152786, "job_id": "AJ_fyxVgrKTusfQ", "timestamp": "2025-11-17T16:47:11.336074+00:00"}refined-appointment-81829
11/17/2025, 9:45 PMproud-match-31207
11/18/2025, 5:29 AMrefined-appointment-81829
11/18/2025, 5:29 AMproud-match-31207
11/19/2025, 5:48 PMAJ_Q5uCDLQoKZpu. The reconnection to LiveKit wasn't successful. Room ID - RM_mhRYGQPWssBb. It tried resuming the conenction but it didn't go through, can you please help check why this can happen? This has caused an issue where the agent was stuck and wasn't responding for 7 mins
The new room session it was trying to connect to - RM_VujmsKk4v6kF.
{"message": "livekit::rtc_engine:453:livekit::rtc_engine - received session close: \"signal client closed: \\\"ping timeout\\\"\" UnknownReason Resume", "level": "WARNING", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T15:57:48.870258+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 0", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T15:57:48.870746+00:00"}
{"message": "_SegmentSynchronizerImpl.resume called after close", "level": "WARNING", "name": "livekit.agents", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T15:58:02.436703+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.240043+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 1", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.240288+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.332808+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 2", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.332983+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.422920+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 3", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.423081+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.515362+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 4", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.515510+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.607453+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 5", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.608397+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.698840+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 6", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.699109+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.791497+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 7", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.791678+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.881934+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 8", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.882097+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.975011+00:00"}
{"message": "livekit::rtc_engine:729:livekit::rtc_engine - resuming connection... attempt: 9", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:33.975199+00:00"}
{"message": "livekit::rtc_engine:731:livekit::rtc_engine - resuming connection failed: signal failure: ws failure: Connection closed normally", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.065930+00:00"}
{"message": "livekit::rtc_engine:663:livekit::rtc_engine - failed to reconnect to the livekit room", "level": "ERROR", "name": "livekit", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.066113+00:00"}
{"message": "failed to publish transcription\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/agents/voice/room_io/_output.py\", line 431, in capture_text\n await self._writer.write(text)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/data_stream.py\", line 295, in write\n await self._send_chunk(chunk_msg)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/data_stream.py\", line 213, in _send_chunk\n raise ConnectionError(cb.send_stream_chunk.error)\nConnectionError: engine: connection error: engine is closed", "level": "WARNING", "name": "livekit.agents", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.067520+00:00"}
{"message": "failed to publish transcription\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/agents/voice/room_io/_output.py\", line 300, in _publish_transcription\n await self._room.local_participant.publish_transcription(transcription)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/participant.py\", line 285, in publish_transcription\n raise PublishTranscriptionError(cb.publish_transcription.error)\nlivekit.rtc.participant.PublishTranscriptionError: engine: connection error: engine is closed", "level": "WARNING", "name": "livekit.agents", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.071639+00:00"}
{"message": "failed to publish transcription\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/agents/voice/room_io/_output.py\", line 300, in _publish_transcription\n await self._room.local_participant.publish_transcription(transcription)\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/participant.py\", line 285, in publish_transcription\n raise PublishTranscriptionError(cb.publish_transcription.error)\nlivekit.rtc.participant.PublishTranscriptionError: engine: connection error: engine is closed", "level": "WARNING", "name": "livekit.agents", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.073389+00:00"}
{"levelname": "INFO", "name": "agent_orchestrator", "process": 272419, "event": "Shutdown Call", "message": "", "pathname": "/home/app/agent-orchestrator/agent_orchestrator/service/call_service.py", "lineno": 1551, "end_reason": "room disconnected", "call_id": "call_AJ_Q5uCDLQoKZpu", "room_name": "tcn_stg_use2_000_inbound_prod_deployment_+15749036527_ptWYTQRqhzLU", "tenant_id": "stg_use2_000", "timestamp": "2025-11-19T16:03:34.074218+00:00"}
{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "room disconnected", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.074274+00:00"}
{"levelname": "INFO", "name": "agent_orchestrator", "process": 272419, "event": "Shutdown Call - Call is Ongoing - Triggering Call End", "end_reason": "room disconnected", "timestamp": "2025-11-19T16:03:34.074358+00:00"}
{"levelname": "INFO", "name": "agent_orchestrator", "process": 272419, "event": "End Call", "end_reason": "room disconnected", "timestamp": "2025-11-19T16:03:34.074457+00:00"}
{"message": "failed to publish transcription\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/livekit/agents/voice/room_io/_output.py\", line 433, in capture_text\n tmp_writer = await self._create_text_writer()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/livekit/agents/voice/room_io/_output.py\", line 405, in _create_text_writer\n return await self._room.local_participant.stream_text(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/participant.py\", line 578, in stream_text\n await writer._send_header()\n File \"/usr/local/lib/python3.11/site-packages/livekit/rtc/data_stream.py\", line 189, in _send_header\n raise ConnectionError(cb.send_stream_header.error)\nConnectionError: engine: connection error: engine is closed", "level": "WARNING", "name": "livekit.agents", "pid": 272419, "job_id": "AJ_Q5uCDLQoKZpu", "timestamp": "2025-11-19T16:03:34.073676+00:00"}
{"levelname": "INFO", "name": "agent_orchestrator", "process": 272419, "event": "Deleting Room", "message": "", "pathname": "/home/app/agent-orchestrator/agent_orchestrator/service/call_service.py", "lineno": 2285, "timestamp": "2025-11-19T16:03:38.277132+00:00"}
{"levelname": "ERROR", "name": "agent_orchestrator", "process": 272419, "event": "Delete Room Error", "message": "", "pathname": "/home/app/agent-orchestrator/agent_orchestrator/service/call_service.py", "lineno": 2307, "error": "TwirpError(code=not_found, message=requested room does not exist, status=404)", "exc_info": "Traceback (most recent call last):\n File \"/home/app/agent-orchestrator/agent_orchestrator/service/call_service.py\", line 2293, in delete_room\n await self.job_ctx.api.room.delete_room(\n File \"/usr/local/lib/python3.11/site-packages/livekit/api/room_service.py\", line 108, in delete_room\n return await self._client.request(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/livekit/api/twirp_client.py\", line 129, in request\n raise TwirpError(\nlivekit.api.twirp_client.TwirpError: TwirpError(code=not_found, message=requested room does not exist, status=404)", "timestamp": "2025-11-19T16:03:38.336024+00:00"}