https://livekit.io logo
Join Slack
Powered by
How do i get the ai to ask question with session.s...
# ask-ai
d
How do i get the ai to ask question with session.say then wait for the users repsonse before contninuing
t
Thanks for reaching out, Billy. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You're asking how to have the agent ask a question with 
session.say
and then wait for the user’s response before continuing. You can rely on LiveKit's built-in turn-taking: after 
session.say
finishes, the agent transitions to listening and will wait for the user's turn (VAD/STT/turn detector will detect when the user speaks). Two common patterns: 1) Let the agent handle turn-taking automatically • Call 
session.say("Question?")
. • After speaking, the agent will listen for the user and you handle the user input in your usual reply generation or event flow (no extra awaits required). Example (Node):
Copy code
await session.say('What is your favorite color?');
// agent automatically listens and waits for user input
2) Use an event listener to explicitly handle the user's reply • Call 
session.say(...)
. • Listen for 
user_input_transcribed
(or the equivalent user input event). • When that event arrives, read 
event.text
and continue. Example (Python):
Copy code
await session.say("What is your favorite color?")

@session.on("user_input_transcribed")
async def on_user_transcribed(event):
    print("User said:", event.text)
    await session.say("Thanks — got it!")
Notes and tips • 
allowInterruptions
can be used when you want the user to interrupt the agent mid-speech: 
session.say(text, { allowInterruptions: true })
. • If you need stricter control (e.g., timeouts, multiple follow-up prompts, or validating the response), implement a timer or state machine: start a timer after 
session.say
, wait for the transcription event, validate the text, and re-prompt if necessary. Would you like a concrete example tailored to your language/runtime or help implementing a timeout/re-prompt flow? Sources: OpenAI Realtime API plugin guide | LiveKit docs | Turn detection and interruptions | LiveKit Docs
d
ok how with this code...for example @function_tool async def finalize_call(self, param_filler: bool = False, user_last_concerns: str = None) -> str: """Finalize call""" logger.info("✅ finalize_call started") session = self.session # Generate summary await self.session.say( "Before I let you go, do you have any questions or concerns?", allow_interruptions=False, ) @session.on("user_input_transcribed") async def on_user_transcribed(event): print("User said:", event.text) await session.say("Thanks — got it!") if user_last_concerns: user_last_concerns = session.userdata.misc logger.info(f"👀 Last Concerns: {user_last_concerns}")
t
You're on the right track. Below is a corrected, minimal pattern you can use in your async Python function to ask the question, wait for the user response, and then continue. Key points: register the event handler before or immediately after asking, capture the response in a local place the function can await, and use an asyncio Future (or Event) so your function pauses until the handler receives the transcription. Example:
Copy code
import asyncio

@function_tool
async def finalize_call(self, param_filler: bool = False, user_last_concerns: str = None) -> str:
    """Finalize call"""
    <http://logger.info|logger.info>("✅ finalize_call started")
    session = self.session

    # Future used to wait for user response
    response_future = asyncio.get_event_loop().create_future()

    @session.on("user_input_transcribed")
    async def on_user_transcribed(event):
        # store/transmit the text and resolve the future
        text = event.text
        print("User said:", text)
        # Optionally save to session userdata
        session.userdata.misc = text
        if not response_future.done():
            response_future.set_result(text)
        # immediate acknowledgement (optional)
        await session.say("Thanks — got it!")

    # Ask the question (agent will listen after speaking)
    await session.say(
        "Before I let you go, do you have any questions or concerns?",
        allow_interruptions=False,
    )

    try:
        # Wait for user response with timeout (adjust seconds as needed)
        user_text = await asyncio.wait_for(response_future, timeout=30.0)
    except asyncio.TimeoutError:
        user_text = None
        await session.say("I didn't hear anything — no problem. Feel free to reach out later.")

    # Continue processing
    if user_text:
        user_last_concerns = user_text
        <http://logger.info|logger.info>(f"👀 Last Concerns: {user_last_concerns}")
        # further logic here

    return "finalized"
Notes: • Using an asyncio Future lets your function pause until the event handler sets the result. • Add a timeout to avoid waiting forever. • 
allow_interruptions=False
prevents the user interrupting while the agent speaks; set to True if you want interruption. • You can store the response in 
session.userdata
if you prefer persistent session state. Would you like this adapted to a different timeout, multi-turn validation (re-prompt on empty), or an example without the immediate acknowledgement inside the handler?
d
Hello?
t
Sorry for the delay — I’m here. Do you want me to: • Walk through the example I sent line‑by‑line with your code, or • Paste a ready-to-run version tailored to your project (same session API, with timeout and storing the reply in 
session.userdata
)? Tell me which you prefer or paste any errors you’re seeing.
2 Views
Open in Slack
PreviousNext
Powered by