Inspect log 16 Mar 2026, 141715.502 Severity Logger livekit.agents Message speech not done in time after interruption, cancelling the speech arbitrarily. Attributes

{speech_id: "speech_d8471d201819"

timeout: 5

code.file.path: "/app/.venv/lib/python3.13/site-packages/livekit/agents/voice/speech_handle.py"

code.function.name: "_on_timeout"

code.line.number: 187n

room_id: "RM_T7eRGcLnYWvQ"

job_id: "AJ_ggCo5QZzXDbp"

logger.name: "livekit.agents"

lk.id: "a9370337-b178-695b-7a83-c7bec2faba27"

}

I am getting this issue, please help me here

Thanks for reaching out, Yethu. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

--- Root Cause Analysis The error you're seeing in LiveKit observability: ▎ "speech not done in time after interruption, cancelling the speech arbitrarily" (timeout: 5s) This comes from the LiveKit SDK's SpeechHandle._on_timeout method. When speech is interrupted and doesn't complete/cancel within 5 seconds, the SDK forcibly kills it. The Problem: end_call.py (line 19-21) await ctx.session.say(message_to_speak, allow_interruptions=False) sleep_duration = (len(message_to_speak) / 15) + 1.5 await asyncio.sleep(sleep_duration) This is the primary culprit. You're calling session.say() inside a function tool and then sleeping. Here's what goes wrong: 1. The LLM decides to call end_call tool 2. session.say() starts playing the closing message 3. If the user speaks (even accidentally), it triggers an interruption 4. But allow_interruptions=False means the speech handle can't be cleanly interrupted 5. The asyncio.sleep() keeps the tool running, blocking the speech handle from resolving 6. After 5 seconds, the SDK force-cancels the speech → "speech not done in time" 7. This can cascade into the agent becoming unresponsive ("not working") Why No Logs in Blob The agent likely crashed or got stuck before reaching the cleanup phase in ResourceCleanupHandler.cleanup_all_resources(). If the speech timeout causes an unhandled cancellation error that propagates up, the shutdown callback chain may not complete properly, meaning _upload_logs_on_failure never runs. Secondary Risk: agent_session.py (line 113) The _setup_user_state_handler also calls session.generate_reply() synchronously when user state changes to "away", which could compound the issue. --- Fix Let me fix the end_call.py tool to handle speech properly: ● Update(tools\end_call.py) ⎿ Added 11 lines, removed 8 lines 15 outcome: "success" or "failure". 16 reason: Internal log reason (e.g., "Verification completed", "User was rude"). 17 """ 18 - # We await it so the text is sent to the TTS engine 19 - await ctx.session.say(message_to_speak, allow_interruptions=False) 18 + logger.info(f"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}") 19 20 - sleep_duration = (len(message_to_speak) / 15) + 1.5 21 - await asyncio.sleep(sleep_duration) 20 + try: 21 + # Say the closing message and wait for playout to complete 22 + speech_handle = ctx.session.say(message_to_speak, allow_interruptions=False) 23 + # Wait for the TTS audio to finish playing out instead of using a crude sleep 24 + await speech_handle.wait_for_playout() 25 + except asyncio.CancelledError: 26 + logger.warning("End call speech was cancelled during playout") 27 + except Exception as e: 28 + logger.error(f"Error during end call speech: {e}") 29 30 - # 3. Log the interaction (for your analytics) 31 - logger.info(f"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}") 32 - 33 - # 4. Initiate shutdown - let framework handle room cleanup 30 + # Initiate shutdown - let framework handle room cleanup 31 job_ctx = get_job_context() 32 if job_ctx: 33 job_ctx.shutdown(reason=f"Agent ended call: {reason}") ● Now let me verify the fix looks correct: ● Read 1 file (ctrl+o to expand) ● Good. Now let me check if the session.say() in the LiveKit SDK actually returns a SpeechHandle that has wait_for_playout(). Let me verify by checking the installed SDK. ● Bash(cd "C:/Users/yethu.krishnan/OneDrive - Provana India Pvt. Ltd/Documents/Provana_Projects/ContactCenterProject/cc-livekitagent/ai-agent" && grep -r "def say" .ve…) ⎿ .venv/Lib/site-packages/livekit/agents/voice/agent_activity.py .venv/Lib/site-packages/livekit/agents/voice/agent_session.py ● Searched for 1 pattern (ctrl+o to expand) ● session.say() returns a SpeechHandle. Let me verify SpeechHandle has wait_for_playout(): ● Searched for 1 pattern (ctrl+o to expand) ● Important - the SDK warns about calling wait_for_playout() from inside a function tool. Let me check the exact guard: ● Read 1 file (ctrl+o to expand) ● The guard at line 146 only blocks if info.speech_handle == self — i.e., you're waiting on the same speech handle that owns the current tool. Since session.say() creates a new speech handle, our call is safe. However, there's still a subtle issue. The say() call from inside a tool creates a new speech that's queued, and the tool's parent speech handle won't complete until the tool function returns. Let me check if RunContext has a wait_for_playout() that's more appropriate: ● Searched for 5 patterns, read 1 file (ctrl+o to expand) ● Now I see the full picture. RunContext.wait_for_playout() waits for the speech prior to the tool running. But what we need is to wait for the new speech created by session.say() inside the tool. The current fix using speech_handle.wait_for_playout() on the new handle returned by say() is correct since it's a different speech handle. However, let me also verify there's no issue with say() being called from within a tool context. Let me check the _on_timeout logic: Searching for 1 pattern, reading 1 file… (ctrl+o to expand) ⎿ "INTERRUPTION_TIMEOUT" what iss your thoughts

Thanks — this is a clear, well-reasoned RCA and fix. I’ll summarize what you’ve found, confirm the correctness of the approach, and add a few small recommendations to make the fix more robust. Acknowledgement • You correctly identified the root cause: calling

session.say()

followed by

asyncio.sleep()

inside a tool with

allow_interruptions=False

blocks cleanup when an interruption occurs, triggering the 5s SpeechHandle timeout and an arbitrary cancel. • Your change to stop using

asyncio.sleep()

and to await the returned

SpeechHandle.wait_for_playout()

is the right direction. Why your fix is correct •

session.say()

returns a

SpeechHandle

. Awaiting

speech_handle.wait_for_playout()

waits for the new speech’s playout to finish rather than sleeping an estimated duration. This avoids keeping the tool alive unnecessarily and lets the SDK handle interruptions cleanly. • The SDK’s guard that prevents awaiting the same speech handle that owns the tool does not block this pattern, since

session.say()

returns a different/new speech handle. So it’s safe to call

wait_for_playout()

for the new handle from inside the tool. • Handling

asyncio.CancelledError

and logging exceptions around

wait_for_playout()

is appropriate and will help avoid unhandled cancellations that can prevent cleanup hooks (like log uploads) from running. Additional recommendations to make the fix more robust • Consider

allow_interruptions=True

when appropriate: ◦ If the closing message can be interrupted by the user (e.g., they try to speak), allowing interruption may reduce race conditions. If you must ensure the message completes for compliance/verification reasons, keep

False

but ensure proper cancellation handling. • Explicitly cancel any in-flight tasks started by the tool on interruption: ◦ If the tool launches background tasks, keep references and call their cancel methods when

speech_handle.interrupted

is set or when

wait_for_playout()

raises

CancelledError

. • Add more logging around interruption state: ◦ Log

speech_handle.interrupted

or when you catch

CancelledError

, including `job_id`/`room_id` to correlate with observability logs. • Ensure cleanup path runs even on unexpected exceptions: ◦ Wrap higher-level tool logic in try/except/finally so cleanup/ResourceCleanupHandler logic isn't skipped by an unhandled exception. • Test these cases explicitly: ◦ Normal flow (no interruption). ◦ Short accidental user input that should interrupt. ◦ Intentional user input immediately after

session.say()

starts. ◦ Tool-level exceptions and cancellation during playout. • Review

min_interruption_duration

/

min_interruption_words

values: ◦ If you see many spurious interruptions, tuning these can reduce false positives or let real interruptions be recognized reliably. Small nit: shutdown placement • Your change calls

job_ctx.shutdown(reason=...)

after speech completion, which is fine. Just confirm the shutdown won’t attempt to await other things that may re-trigger speech or otherwise race with finalization. If shutdown triggers more speech, ensure those flows are interruption-safe. If you want, I can: • Review a sanitized snippet of the updated

end_call.py

to confirm exception and cancellation handling is covered end-to-end. • Suggest unit or integration test scenarios (with steps) you can run to validate the fix in your environment. Nice work on the RCA and the fix — it addresses the core problem and should prevent the 5s forced cancels once the cancellation/wait handling and logging are in place.

if I allow interuption but still it will delete the room right

Good question — short answer: allowing interruptions only affects whether the speech can be stopped cleanly; it does not automatically stop your shutdown/room-deletion logic. If your tool still calls the shutdown flow after the speech finishes (or after you call

job_ctx.shutdown()

), the room will be deleted regardless of whether the speech was interrupted. More detail and what to do • Allowing interruptions (

allow_interruptions=True

) lets the SDK cancel the speech cleanly when the user speaks, avoiding the 5s forced timeout and arbitrary cancel. That prevents the timeout error you saw. • Room deletion (your call to

job_ctx.shutdown(...)

or other cleanup code) is controlled by your application logic, not by

allow_interruptions

. If you call shutdown unconditionally after

wait_for_playout()

(or after

session.say()

), the room will be deleted even when the user interrupted the speech. How to ensure the desired behavior • Decide whether an interruption should abort the end-call sequence or still lead to shutdown. ◦ If interruption should cancel the shutdown: check

speech_handle.interrupted

or catch

asyncio.CancelledError

and skip or postpone

job_ctx.shutdown(...)

. ◦ If interruption should NOT stop shutdown (i.e., you always close the call): keep the shutdown call but ensure it runs after you properly handle/catch cancellation so cleanup completes gracefully. Recommended pattern 1. Start the speech and capture the handle: ◦

speech_handle = ctx.session.say(..., allow_interruptions=True)

1. Await playout with error handling: ◦

try: await speech_handle.wait_for_playout()

◦

except asyncio.CancelledError: <http://logger.info|logger.info>("speech interrupted")

◦

except Exception as e: logger.error(...)

1. Make shutdown conditional or unconditional based on interruption: ◦ If conditional: ▪︎

if not speech_handle.interrupted: job_ctx.shutdown(...)

▪︎ else: log, take alternative action (e.g., retry, prompt user, or schedule shutdown) ◦ If unconditional: ▪︎ call

job_ctx.shutdown(...)

but ensure any cancellation exceptions are handled so cleanup runs. Extra tips • If you need to ensure the user’s interruption (e.g., they want to continue interacting) prevents deletion, use the conditional path and perhaps add a short grace period or confirmation before shutdown. • Add clear logging when you decide to skip or proceed with shutdown so you can correlate with observability logs (include `room_id`/`job_id`). • Test both flows: interrupting immediately and not interrupting to confirm shutdown behavior matches expectations. If you tell me which behavior you want (interrupt = cancel shutdown, or interrupt = still delete room), I can give a concise code snippet showing the exact conditional logic to implement.

If I wanted to do this how can I means if interupt then it should not delete

Thanks — good choice to make shutdown conditional. Below is a concise, practical pattern you can drop into your tool (e.g., end_call.py) so that if the closing speech is interrupted, you skip deleting the room. What this does • Starts the closing speech and captures the returned

SpeechHandle

. • Waits for playout with proper error handling. • If

speech_handle.interrupted

is true (or

wait_for_playout()

raises

CancelledError

), it skips

job_ctx.shutdown(...)

and logs the decision. • If not interrupted, it proceeds with shutdown. Suggested code pattern (Python, asyncio):

# inside your end_call tool
<http://logger.info|logger.info>(f"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}")

# Start speech and get the new SpeechHandle
speech_handle = ctx.session.say(message_to_speak, allow_interruptions=True)

try:
    # Wait for the TTS audio to finish playing out
    await speech_handle.wait_for_playout()
except asyncio.CancelledError:
    <http://logger.info|logger.info>("End-call speech was cancelled during playout (CancelledError). Will NOT proceed to shutdown.")
    # Skip shutdown because user interrupted
    return {"outcome": "interrupted", "reason": "speech_cancelled_by_user"}
except Exception as e:
    logger.error(f"Error while waiting for playout: {e}")
    # Decide policy on errors: conservative approach is to skip shutdown or perform safe shutdown
    return {"outcome": "failure", "reason": f"playout_error: {e}"}

# After successful playout, check interrupted flag as a safety net
if getattr(speech_handle, "interrupted", False):
    <http://logger.info|logger.info>("Speech handle indicates interruption. Skipping shutdown.")
    return {"outcome": "interrupted", "reason": "speech_handle_interrupted"}

# No interruption: proceed with shutdown
job_ctx = get_job_context()
if job_ctx:
    <http://logger.info|logger.info>("Speech completed without interruption. Initiating shutdown.")
    job_ctx.shutdown(reason=f"Agent ended call: {reason}")

return {"outcome": "success", "reason": "call_ended_normally"}

Notes and best practices • Use

allow_interruptions=True

so interruptions are detected and handled cleanly. If you must keep

False

for regulatory reasons, you’ll need extra cancellation logic—this pattern assumes interruptions are allowed. • Catch

asyncio.CancelledError

specifically since cancellations from the runtime will raise that. • Check

speech_handle.interrupted

after playout to cover any races where interruption occurred but didn’t raise during

wait_for_playout()

. • If your tool spawns background tasks, cancel them when skipping shutdown so nothing leaks. • Log `room_id`/`job_id` with messages to help correlate with observability traces. • Consider a short grace period (e.g., a small debounce) if you want to avoid skipping shutdown for accidental, very brief interruptions. If you want, paste a sanitized snippet of your current end_call tool and I’ll adapt this pattern directly into it (including the exact variable names you use).

# end_call.py import asyncio from livekit.agents import RunContext, function_tool, get_job_context from loguru import logger @function_tool async def end_call(ctx: RunContext, message_to_speak: str, outcome: str, reason: str): """ Only ends the call when it is necessary and you are confident about the ending scenario. Args: message_to_speak: A natural, context-aware closing statement generated specifically for this conversation. Do NOT use generic phrases like "Goodbye" or "I have to run." Instead, reference the specific topic discussed (e.g., "I'll get that email over to you right away, have a great afternoon" or "Thanks for clarifying those details about the appointment"). outcome: "success" or "failure". reason: Internal log reason (e.g., "Verification completed", "User was rude"). """ logger.info(f*"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}"*) try: # Say the closing message and wait for playout to complete speech_handle = ctx.session.say(message_to_speak, _allow_interruptions_*=False*) # Wait for the TTS audio to finish playing out instead of using a crude sleep await speech_handle.wait_for_playout() except asyncio.CancelledError: logger.warning("End call speech was cancelled during playout") except Exception as e: logger.error(f*"Error during end call speech: {e}"*) finally: # Always initiate shutdown - ensures cleanup (log upload, egress stop, room deletion) runs job_ctx = get_job_context() if job_ctx: job_ctx.shutdown(_reason_*=*f*"Agent ended call: {reason}"*) is this fine

Thanks — you’re very close. I reviewed your pasted

end_call.py

and it currently always shuts down in the

finally

block, so it will delete the room even when the speech was interrupted. Below I explain why that happens and show a small, safe change that implements the behavior you asked for: if the closing speech is interrupted, skip the shutdown. Why your current code still deletes the room • The

finally

block always runs whether

wait_for_playout()

completed normally, raised

CancelledError

, or any other exception. Because you call

job_ctx.shutdown(...)

inside

finally

, the room will be deleted even when speech was cancelled. Recommended changes (concise) • Allow interruptions on the TTS so the SDK can detect them cleanly. • Await the returned

SpeechHandle.wait_for_playout()

and handle

CancelledError

. • Move shutdown into the non-interrupted success path (or make it conditional based on

speech_handle.interrupted

). • Return a result when skipping shutdown so the caller can act accordingly and you log the decision. Here’s an adapted version of your tool implementing this pattern:

import asyncio
from livekit.agents import RunContext, function_tool, get_job_context
from loguru import logger

@function_tool
async def end_call(ctx: RunContext, message_to_speak: str, outcome: str, reason: str):
    """
    Ends the call unless the closing speech was interrupted by the user.
    """
    <http://logger.info|logger.info>(f"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}")

    # Start speech with interruptions allowed so we can detect user speech
    speech_handle = ctx.session.say(message_to_speak, allow_interruptions=True)

    try:
        await speech_handle.wait_for_playout()
    except asyncio.CancelledError:
        <http://logger.info|logger.info>("End-call speech was cancelled during playout. Skipping shutdown.")
        # Optional: cancel any tool-local background tasks here
        return {"outcome": "interrupted", "reason": "speech_cancelled_by_user"}
    except Exception as e:
        logger.error(f"Error during end call speech playout: {e}")
        # Conservative choice: skip shutdown so we don't accidentally delete the room on error
        return {"outcome": "failure", "reason": f"playout_error: {e}"}

    # Safety check: if interrupted flag is set, skip shutdown
    if getattr(speech_handle, "interrupted", False):
        <http://logger.info|logger.info>("Speech handle indicates interruption. Skipping shutdown.")
        return {"outcome": "interrupted", "reason": "speech_handle_interrupted"}

    # No interruption: proceed with shutdown
    job_ctx = get_job_context()
    if job_ctx:
        <http://logger.info|logger.info>("Speech completed without interruption. Initiating shutdown.")
        job_ctx.shutdown(reason=f"Agent ended call: {reason}")

    return {"outcome": "success", "reason": "call_ended_normally"}

Additional suggestions • If your tool spawns background tasks, explicitly cancel them when you skip shutdown. • Add room_id/job_id to logs for traceability. • Consider a short debounce (e.g., 0.5s) before deciding interruption should skip shutdown if you want to ignore accidental, very brief user noises. • Test these flows: no interruption, immediate interruption, and playout error. If you want, paste any background tasks you spawn in this tool and I’ll show exactly where to cancel them when skipping shutdown.

it is goo dbut can we send the user asked chunk in the context as an interupted user want to ask something like that then that chunk also?

Great — yes, you can both (A) skip shutdown when speech is interrupted and (B) capture the chunk of user speech that caused the interruption and feed it back into your flow. Below I’ll show a concise pattern you can drop into your

end_call.py

tool and explain how the interrupting input is surfaced. Quick summary of behavior • When the user speaks during agent playback, LiveKit stops playback, begins STT, and that transcribed user input is added to the conversation as a user turn. • You can detect the interruption using the returned

SpeechHandle

(

speech_handle.interrupted

) or by catching

asyncio.CancelledError

from

wait_for_playout()

. • To avoid deleting the room when interrupted, skip

job_ctx.shutdown(...)

when the handle indicates interruption. • The user’s transcribed chunk is available in the session/chat context (and via events/hooks) so you can include it in the follow-up prompt or tool logic. Pattern to implement (adapt this into your tool) 1. Start the closing speech with interruptions allowed and capture the handle. 2. Await playout and catch cancellations. 3. If interrupted, read the interrupted user input from the chat context or event, and return/use it rather than shutting down. 4. If not interrupted, proceed with shutdown. Example adapted for your

end_call.py

(keeps your names & style):

import asyncio
from livekit.agents import RunContext, function_tool, get_job_context
from loguru import logger

@function_tool
async def end_call(ctx: RunContext, message_to_speak: str, outcome: str, reason: str):
    <http://logger.info|logger.info>(f"Call Ending | Outcome: {outcome} | Reason: {reason} | Spoke: {message_to_speak}")

    # 1) Start speech and allow interruptions so we can detect and capture user's chunk
    speech_handle = ctx.session.say(message_to_speak, allow_interruptions=True)

    try:
        # 2) Wait for playout; CancelledError will be raised if interrupted
        await speech_handle.wait_for_playout()
    except asyncio.CancelledError:
        <http://logger.info|logger.info>("End-call speech was cancelled during playout. Will NOT proceed to shutdown.")

        # 3) Retrieve the interrupting user chunk from the session/chat context
        # The user input should already be transcribed and added to the chat context after turn completion.
        # Access the latest user message (sanitized to your schema). Example approach:
        interrupted_text = None
        try:
            # ctx.session.chat_ctx or similar holds conversation messages; pick latest user message
            chat_ctx = ctx.session.chat_ctx
            # Walk messages from newest to oldest to find the most recent user message
            for msg in reversed(chat_ctx.messages):
                if msg.role == "user":
                    interrupted_text = msg.text_content
                    break
        except Exception:
            logger.exception("Failed to read interrupted user text from chat context.")

        # Return the interrupted chunk so the caller can process it (or you can call generate_reply)
        return {
            "outcome": "interrupted",
            "reason": "speech_cancelled_by_user",
            "interrupted_user_text": interrupted_text,
        }
    except Exception as e:
        logger.error(f"Error during end call speech playout: {e}")
        return {"outcome": "failure", "reason": f"playout_error: {e}"}

    # 4) Safety check after successful playout
    if getattr(speech_handle, "interrupted", False):
        <http://logger.info|logger.info>("Speech handle indicates interruption. Skipping shutdown.")
        # Fetch interrupted user input similarly if needed
        return {"outcome": "interrupted", "reason": "speech_handle_interrupted", "interrupted_user_text": None}

    # 5) No interruption: proceed with shutdown
    job_ctx = get_job_context()
    if job_ctx:
        <http://logger.info|logger.info>("Speech completed without interruption. Initiating shutdown.")
        job_ctx.shutdown(reason=f"Agent ended call: {reason}")

    return {"outcome": "success", "reason": "call_ended_normally"}

Notes on where the interrupted chunk comes from • LiveKit records and transcribes the user audio that interrupts playback. That transcription is added to your conversation/chat context as a user message once the system completes the turn. • You can access it in multiple ways: ◦ Read the latest user message from

ctx.session.chat_ctx.messages

(example in snippet). ◦ Use the

user_input_transcribed

event handler to process the transcript as it arrives. ◦ Inspect

on_user_turn_completed

hook if you want to mutate or route the interrupted text before it reaches the LLM. Best-practice tips • Include

room_id

and

job_id

in your logs to correlate interruptions with observability traces. • If you spawn background tasks inside the tool, cancel them when skipping shutdown to avoid leaks. • Optionally add a short debounce (e.g., 0.5s) before skipping shutdown to ignore accidental micro-noise interruptions. • If you want the agent to immediately respond to the interrupted user chunk, call

ctx.session.generate_reply()

after capturing the text so the conversation continues naturally. If you’d like, paste a sanitized sample of your

chat_ctx.messages

structure (or confirm how you store messages) and I’ll tailor the snippet to extract the interrupted chunk exactly as your project represents messages. Sources: Agent speech and audio | LiveKit Documentation | Turns overview | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | Function tool definition | LiveKit Documentation