This message was deleted.

Thanks for reaching out, Aleh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

After

wait_for_playout()

completes, inspect

self.session.current_speech.interrupted

(or the equivalent

SpeechHandle

flag). If it’s true, you can decide whether to: ▪︎ Re-issue a short closing message (if you want to acknowledge the user) i want it to continue from where it was interupted

Thanks — I understand you want the agent to continue from the exact point where playback was interrupted rather than stopping or restarting the whole session. I’ll be direct: LiveKit Agents do not support resuming a TTS audio stream mid-playback. The system marks the speech as interrupted, truncates the transcript/context to what was actually played, and does not expose an exact playback offset to resume from. What you can do instead (how to simulate resumption) • Track the full text you intended to speak before starting playback (store the original string and sensible split points like sentences or clauses). • Detect interruption after

wait_for_playout()

by checking

SpeechHandle.interrupted

(or using the

playback_finished

event). • Reconstruct the remaining portion to speak and re-synthesize it. Options to determine the remaining portion: ◦ Use sentence/phrase boundaries and assume the interruption occurred at the last completed boundary. ◦ Use the last few words actually heard (if you have a short transcript of played audio) and ask the LLM to continue from there. • Prompt the LLM to resume naturally rather than trying to stitch exact audio offsets. Example prompt: "You were interrupted while saying: '[last 6–10 words]'. Continue the message from that point." Suggested implementation outline 1. Before speaking, save the message and split into segments (sentence or clause boundaries). 2. Play the message and await playout: a. handle = await session.say(full_message) b. await handle.wait_for_playout() 1. After playout completes, check interruption: ◦ If not interrupted: proceed to room teardown. ◦ If interrupted: ▪︎ Determine which segment(s) were not delivered (e.g., using the last confirmed sentence). ▪︎ Ask the LLM to produce the remaining text or programmatically pick the remaining segments. ▪︎ Call

session.say(remaining_text)

to continue. 1. Log events (played vs interrupted, timestamps, last heard words) to improve heuristics over time. Practical tips and tradeoffs • There’s no API to get an exact playback offset in seconds, so exact byte-level resumption isn’t possible. • Re-synthesizing introduces extra latency; keep resumed segments short to reduce perceived disruption. • For critical closing lines, consider

allow_interruptions=False

so they’re not cut off. • If you want more natural continuation, let the LLM generate the continuation using the last spoken fragment as context — this often sounds more fluid than trying to piece back an exact remainder. If you’d like, I can draft a concise code example that: • Saves the original message, • Detects interruption, • Uses a simple sentence-boundary approach to compute the remaining text, • Re-asks the LLM to continue from the last words and replays the continuation. Which would you prefer? • A code snippet that programmatically slices the original message and replays remaining text, or • A snippet that prompts the LLM to generate a natural continuation using the last few words? Sources: How to detect when an agent has finished speaking | LiveKit | How to detect when an agent has finished speaking | Agent speech and audio | LiveKit Documentation

A snippet that prompts the LLM to generate a natural continuation using the last few words?

Thanks — I hear you want a concise, working example that prompts the LLM to continue naturally from the last few words when the agent is interrupted. Below is a focused snippet and a brief explanation you can drop into your agent. It: • Detects interruption via the speech handle • Extracts the last few heard words from the synchronized transcript (or from the assistant message) • Prompts the LLM to continue naturally from those last words • Replays the continuation via TTS Code snippet (concise)

import asyncio
from livekit.agents import Agent, AgentSession, JobContext, cli, inference
from livekit.agents.llm import ChatContext, ChatMessage

class ContinuationAgent(Agent):
    async def on_user_turn_completed(self, user_message):
        # Start generating the reply and get the speech handle
        speech_handle = await self.session.generate_reply()
        if speech_handle:
            # Monitor completion asynchronously
            asyncio.create_task(self._on_speech_done(speech_handle))

    async def _on_speech_done(self, handle):
        await handle.wait_for_playout()

        # Was playback interrupted?
        if not getattr(handle, "interrupted", False):
            return  # no interruption — nothing to resume

        # Try to get what was actually heard (synchronized transcript if available)
        interrupted_text = getattr(handle, "synchronized_transcript", None) or ""

        # Fallback: if no synchronized transcript, attempt to extract from chat items
        if not interrupted_text:
            chat_items = getattr(handle, "chat_items", []) or []
            for item in reversed(chat_items):
                if item.type == "message" and item.role == "assistant":
                    interrupted_text = item.text_content or ""
                    break

        interrupted_text = interrupted_text.strip()
        if not interrupted_text:
            # Nothing to base continuation on; optionally ask the user a clarification
            await self.session.say("I was interrupted and couldn't determine what was heard. Could you tell me to continue?")
            return

        # Extract last few words to use as context (adjust count as desired)
        words = interrupted_text.split()
        last_words = " ".join(words[-5:]) if len(words) >= 5 else interrupted_text

        # Build a succinct continuation prompt for the LLM
        continuation_instructions = (
            f"You were speaking and got interrupted. The user heard: '{interrupted_text}'. "
            f"Your last words were: '{last_words}'. Continue the message naturally from where you left off. "
            "Do not repeat what was already said; pick up and finish the thought in a conversational tone."
        )

        # Optionally play a short bridging phrase (keeps UX smooth)
        await self.session.say("Let me continue where I left off...", allow_interruptions=False)

        # Generate the continuation using the LLM and speak it
        # generate_reply will run the LLM and then TTS playback
        await self.session.generate_reply(instructions=continuation_instructions, allow_interruptions=True)

# Entrypoint / server wiring (minimal)
server = AgentServer()

@server.rtc_session(agent_name="continuation-agent")
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        stt=inference.STT("deepgram/nova-3"),
        llm=inference.LLM("openai/gpt-4.1-mini"),
        tts=inference.TTS("cartesia/sonic-3"),
        use_tts_aligned_transcript=True,  # helps populate synchronized_transcript
    )

    await session.start(agent=ContinuationAgent(), room=ctx.room)
    await ctx.connect()

if __name__ == "__main__":
    cli.run_app(server)

Notes, tips and tradeoffs • There is no native API to resume audio at an exact byte/second offset. This approach asks the LLM to generate a natural continuation using the last few heard words as context. •

synchronized_transcript

(when enabled via

use_tts_aligned_transcript=True

) is the best source for what the user actually heard. If that’s not available,

chat_items

can provide the assistant’s message text. • Keep resumed segments short to minimize latency and avoid repeating content. • For very critical lines, consider

allow_interruptions=False

so they’re not cut off. • You can tune how many last words you pass to the LLM (3–8 words usually works well) and adjust the bridging phrase to match your voice UX. If you want, I can: • Provide a variant that uses sentence-boundary slicing instead of last-n-words, or • Expand the continuation prompt to include additional context (last user utterance, intent, or conversation history) for even smoother continuity. Which would you like next? Sources: livekit-agents/livekit/agents/voice/agent_activity.py | Text and transcriptions | LiveKit Documentation

i use sts and has no on_user_turn_completed event

Thanks — I hear you: you’re using STT and don’t have an

on_user_turn_completed

hook. Below is a concise, STT-compatible approach that detects an interruption via the active speech handle, extracts the last few heard words (using synchronized transcript when available), prompts the LLM to continue naturally from those words, and plays the continuation. Key idea • Use the active

SpeechHandle

(or

session.current_speech

) to detect interruption. • Pull the synchronized transcript (what the user heard) or fallback to the assistant message text. • Prompt the LLM with the last few words and an instruction to continue without repeating. • Speak the LLM-generated continuation. Concise code snippet (STT / no on_user_turn_completed)

import asyncio
from livekit.agents import Agent, AgentSession, JobContext, cli, inference

class ContinuationAgent(Agent):
    async def on_user_transcript_final(self, transcript_text: str):
        # Called when your STT layer reports a final user transcript
        # Normal flow: generate reply for the user transcript
        speech_handle = await self.session.generate_reply(user_input=transcript_text)

        if speech_handle:
            asyncio.create_task(self._monitor_and_resume(speech_handle))

    async def _monitor_and_resume(self, handle):
        # Wait until playout completes (or interruption occurs)
        await handle.wait_for_playout()

        # If playback wasn't interrupted, nothing to resume
        if not getattr(handle, "interrupted", False):
            return

        # Prefer synchronized transcript (what user actually heard)
        interrupted_text = getattr(handle, "synchronized_transcript", "") or ""

        # Fallback: if synchronized transcript missing, look into chat items
        if not interrupted_text:
            chat_items = getattr(handle, "chat_items", []) or []
            for item in reversed(chat_items):
                if getattr(item, "type", "") == "message" and getattr(item, "role", "") == "assistant":
                    interrupted_text = getattr(item, "text_content", "") or ""
                    break

        interrupted_text = interrupted_text.strip()
        if not interrupted_text:
            # If nothing is available to base a continuation on, ask the user to clarify
            await self.session.say("I was interrupted and couldn't tell what you heard. Could you ask me to continue?")
            return

        # Extract the last few words to use as LLM context
        words = interrupted_text.split()
        last_words = " ".join(words[-5:]) if len(words) >= 5 else interrupted_text

        continuation_instructions = (
            f"You were speaking to the user and got interrupted. The user heard: '{interrupted_text}'. "
            f"Your last words were: '{last_words}'. Continue the message naturally from that point. "
            "Do not repeat what was already said; pick up the thought and finish it conversationally."
        )

        # Optional bridging phrase for UX
        await self.session.say("Let me continue where I left off...", allow_interruptions=False)

        # Ask the LLM to generate a continuation and speak it
        await self.session.generate_reply(instructions=continuation_instructions, allow_interruptions=True)

How to wire this in an STT setup • Replace

on_user_transcript_final

with whatever event your STT layer provides when it emits a final transcript (many STT integrations emit a final result callback). • Ensure your AgentSession is started with

use_tts_aligned_transcript=True

if your setup supports it, so

synchronized_transcript

is populated. • If your STT provider already supplies a “final transcription” callback, call

on_user_transcript_final(transcript)

from that callback. Practical tips • There’s no API to resume audio at an exact byte/second offset — this approach uses the LLM to produce a natural continuation from the last-heard words. • If

synchronized_transcript

is available, it’s the best source of what the user actually heard; otherwise use assistant message text. • Keep resumed segments short to reduce latency. • For critical lines, set

allow_interruptions=False

to avoid being cut off. If you tell me: • the exact STT callback/event name you do have (e.g.,

on_transcript_final

,

user_input_transcribed

, etc.), and • whether you can enable

use_tts_aligned_transcript

, I’ll adapt the snippet to your event names and show how to wire the STT callback into the agent. Sources: Turns overview | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | Text and transcriptions | LiveKit Documentation