We want the LLM to call tools async in the backgro...
# ask-ai
m
We want the LLM to call tools async in the background. We are using the AsyncRunContext and await context.update("") to hand over control to the LLM. But the LLM decides what to say before it decides to call a tool. So for async tools, the natural flow is: LLM speaks ("I'll look that up for you"), then calls the tool. By the time the tool runs in the background and releases the LLM to talk, it has nothing left to say — it already said its piece. We want the opposite: call the tool first, then talk while it executes. But the LLM has no way to know from the tool schema alone that it should suppress its instinct to narrate before acting.
✅ 1
t
Thanks for reaching out, Andreas. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I hear you: you want the LLM to invoke async tools first (so the tool runs in background) and only then narrate progress/results, but the LLM currently speaks before calling the tool. Short answer: LiveKit does not provide a built‑in mechanism that forces the model to call an async tool before it generates its initial narration. The model decides action order; the tool schema alone won’t suppress its instinct to narrate first. What you can do now (practical workarounds): • Change the agent prompt/instructions — add an explicit rule such as: “On user requests requiring a lookup, immediately call the matching async tool and do not produce any user-facing text until the tool returns progress via
ctx.update()
.” • Make tool intent explicit — craft the tool name + description to clearly state it must be invoked immediately (e.g., “ImmediateBackgroundLookup: call immediately, then report progress via updates”). • Use pre-turn retrieval — if applicable, run your lookup before handing control to the LLM (perform RAG or a prior fetch) so the LLM has the data available and won’t narrate a lookup intent. • Rely on
AsyncRunContext
updates
— once the tool runs, use
await ctx.update("...")
frequently; the agent will translate those updates into speech while the tool continues. • Combine approaches — small prompt rule + explicit tool description + pre-fetch for critical paths gives the best chance the LLM will call the tool first. If you’d like, tell me: • which LLM (model) you’re using, and • one example of the tool signature + current prompt you give the agent. With that I can suggest a concrete prompt rewrite and a sample tool description to increase the chance the model calls the tool before speaking. Sources: Async tools | LiveKit Documentation | External data and RAG | LiveKit Documentation | The ReAct Pattern for Voice Agents and How AI Agents Think, Act, and Respon..., with interactive elements