what is the recommended way to handle long running...
# ask-ai
a
what is the recommended way to handle long running function calls by an agent? behaviour we want - agent says "let me look into that for you....[function call dispatched]....[conversation continues on other topics]....[response received]... right, so based on the results...." python agent standard stt llm tts pipeline
✅ 1
t
Thanks for reaching out, Dima. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking how to handle long-running function calls in a Python STT → LLM → TTS agent so the agent can acknowledge the request, dispatch work in the background, continue the conversation, then later report results. Below is the recommended pattern and key practices. Recommended pattern (concise) 1. Start the long-running work as a background asyncio task (so it doesn't block the agent). 2. Immediately give the user a verbal/status acknowledgement with
session.generate_reply()
(e.g., “Let me look into that for you…”). 3. Let the agent continue accepting/handling other turns while the background task runs. 4. When the task completes, deliver the result by: ◦ Sending a follow-up reply (TTS) when appropriate, or ◦ Storing result state and notifying the user via an event/RPC to the UI if you want non-verbal notification first. 1. Handle interruptions, cancellations, and errors cleanly (cancel the task if needed and raise a ToolError to surface failures). Minimal example sketch
Copy code
async def handle_request(context, query):
    # 1) start background job
    bg = asyncio.ensure_future(do_long_work(query))

    # 2) immediate verbal acknowledgement
    await context.session.generate_reply(instructions="Let me look into that for you.")

    # 3) continue processing other conversation turns while bg runs

    # Later: await result (in another handler/turn) and respond
    result = await bg
    await context.session.generate_reply(instructions=f"Right — based on the results: {result}")
Best practices • Use
context.speech_handle.wait_if_not_interrupted()
if you need to synchronize speech with tool state or cancel if the user interrupts. • For mutating operations, call
context.disallow_interruptions()
while performing the critical section to avoid partial state. • Provide periodic or continuous feedback for very long ops (short status TTS, "thinking" audio, or UI progress via RPC). • Pre-synthesize common acknowledgement TTS to reduce perceived latency. • Always surface failures gracefully (raise a
ToolError
or send a clear error reply). If you want, I can: • Convert the sketch into a full runnable example tied to your agent framework, • Show how to notify a web UI via LiveKit RPC when results are ready, • Or add interruption/cancellation handling for your specific use case. Which would be most helpful? Sources: Function tool definition | LiveKit Documentation | External data and RAG | LiveKit Documentation | https://docs.livekit.io/agents/logic/tools/definition.md| | Bring your LangChain Agents to LiveKit | LiveKit, with interactive elements
a
is the new
AsyncToolset
what i need?
t
I don't have any reference to an
AsyncToolset
in the docs I can access. It may be a new/experimental API or a library-specific helper outside the materials I see. To answer whether it’s what you need, can you tell me: • Where you found
AsyncToolset
(link or repo/package name)? • Do you need background task scheduling only, or also built-in interruption, progress updates, and UI/RPC notifications? • Any constraints (must use pure asyncio, run inside the agent process, or be cancellable from the client)? If you share that, I’ll confirm whether it fits your long-running call pattern and either show how to use it or map the recommended asyncio +
session.generate_reply()
pattern to it. Sources: AsyncAI TTS plugin guide | LiveKit Documentation | An open source stack for real-time multimodal AI | LiveKit | Voice Agent Architecture: STT, LLM, and TTS Pipelines Explained | LiveKit | https://docs.livekit.io/agents/logic/tools/definition.md| | Function tool definition | LiveKit Documentation, with interactive elements