https://livekit.io logo
Join Slack
Powered by
is there a way to do the verbal status updates whe...
# ask-ai
r
is there a way to do the verbal status updates when making the LLM call?
✅ 1
t
Thanks for reaching out, Rithvik. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking whether you can provide verbal status updates while making an LLM call. Yes—you can. What you can do: • Use 
session.generate_reply()
from inside a tool/agent to speak short status messages (e.g., “Let me check that for you”) while the LLM work runs. • For minimal latency, pre-synthesize and cache common TTS phrases and play those immediately. • Start the verbal update without awaiting it, run the LLM call concurrently, then cancel/interrupt the hold message if the LLM finishes quickly: a. Start hold speech (or `say()`/`generate_reply()` returning a handle). b. Call the LLM (await). c. If the hold speech is still playing, interrupt/cancel it. • If you need the speech to be uninterruptible (for critical write operations), call 
context.disallow_interruptions()
at the start of the tool. If you want, tell me which environment/language you’re using (Python/JS, agent framework, or pipeline) and I’ll provide a short code snippet matching your setup. Sources: External data and RAG | LiveKit Documentation | Audio customization | LiveKit Documentation | Agent speech and audio | LiveKit Documentation, with interactive elements
r
I am using python
t
Thanks — below is a concise Python example showing how to play a verbal "hold" message while an LLM call runs, then interrupt the hold if the LLM finishes early.
Copy code
# Example uses an async agent/tool context similar to LiveKit-style APIs:
# - context.session.say(...) returns a handle object with .done() and .interrupt()
# - context.session.generate_reply(...) can also be used for natural replies
# - call_llm() represents your long-running LLM call

import asyncio

async def call_llm():
    # simulate long-running LLM call
    await asyncio.sleep(3)
    return "LLM result"

async def run_with_verbal_status(context):
    # Optionally use cached TTS audio for instant playback:
    # cached_audio = get_cached_audio("Let me check that for you")
    # hold_handle = context.session.say("Let me check that for you.", audio=cached_audio)

    # Start a spoken hold message without awaiting it
    hold_handle = context.session.say("One moment while I check that for you.")

    # Run the LLM call concurrently
    llm_task = asyncio.create_task(call_llm())

    # Option A: Wait for the LLM result and stop hold early if still playing
    result = await llm_task
    if not hold_handle.done():
        hold_handle.interrupt()  # stop the hold message immediately

    # Option B: If you want to allow user speech to interrupt automatically, do nothing special.

    # Send final spoken reply (or use session.generate_reply for a reply from the agent)
    await context.session.generate_reply(instructions=f"Here is the result: {result}")
    return result
Notes: • Pre-synthesizing/caching common phrases reduces TTS latency. • If you must prevent interruption during the tool, call 
context.disallow_interruptions()
at the start. • Use 
generate_reply()
when you want the agent to produce a natural spoken reply after the LLM completes. If you’d like, I can adapt this to your exact agent framework or show a version using 
generate_reply()
for the hold message or cached audio usage. Which do you prefer? , with interactive elements
2 Views
Open in Slack
PreviousNext
Powered by