https://livekit.io logo
Join Slack
Powered by
how do I intercept the LLM response OR errors like...
# ask-ai
a
how do I intercept the LLM response OR errors like 429 and handle them by saying something to user?
✅ 1
t
Thanks for reaching out, Hiren. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking how to intercept LLM responses and handle errors like HTTP 429 so you can say something to the user. Below are two practical patterns you can use with LiveKit agents. 1) Raise a ToolError inside your tool to return a user-facing message • In any function tool, catch the failure (e.g., a 429) and raise 
ToolError
with the message you want the LLM to relay to the user. • Example (Python):
Copy code
from livekit.agents.llm import ToolError

@function_tool()
async def my_tool(context, ...):
    try:
        # call to LLM or external API
    except SomeHttp429Error:
        raise ToolError("I'm rate-limited right now — please try again in a moment.")
• Effect: the agent receives the 
ToolError
and will respond to the user with that text, letting you present a friendly, controlled message. 2) Use FallbackAdapter + in-tool session messages for retries or backup LLMs • Configure 
llm.FallbackAdapter
with a primary LLM and one or more fallbacks so if the primary returns 429 the call can automatically route to the next provider. • While retrying or when calls may take long, use 
context.session.say("Checking that now…")
(from inside the tool) to provide immediate user feedback. • This combination gives higher availability and better UX during transient errors. When to use which: • Use 
ToolError
when you want the agent to explicitly tell the user a specific failure or instruction (e.g., “please try again in 30 seconds”). • Use 
FallbackAdapter
when you want automatic failover and fewer visible error messages; combine with 
context.session.say
for progress updates. If you want, tell me: • Which SDK/language you’re using (Python, JS), and • Whether you prefer automatic failover or explicit user-facing error messages — and I’ll give a concrete code snippet tailored to your stack. Sources: The ReAct Pattern for Voice Agents and How AI Agents Think, Act, and Respon... | Sequential Pipeline Architecture for Voice Agents | LiveKit | Function tool definition | LiveKit Documentation | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Fallback strategies | LiveKit Documentation, with interactive elements
a
I want to handle it in overall main pipeline of AgentSession and not inside the function tool
t
Thanks — you're asking how to handle LLM responses/errors (like 429) centrally at the AgentSession pipeline level instead of inside each function tool. Short answer: handle this with AgentSession-level events plus the Agent/LLM FallbackAdapter. AgentSession emits error/availability events you can listen to, and the Agent FallbackAdapter provides pipeline-level automatic failover for LLMs (it marks providers unhealthy on errors like 429 and switches to fallbacks). You can also emit session messages (e.g., 
context.session.say(...)
) or send a user-facing response from your session event handler when an error occurs. Quick next steps so I can give a concrete example: • Which SDK/language are you using (Python or JavaScript)? • Do you want automatic failover (fallbacks) or explicit user-facing messages when a 429 occurs? Tell me those and I’ll return a concise code snippet showing where to listen for session errors and how to wire an 
llm.FallbackAdapter
(and optionally send a user message from the AgentSession). Sources: Fallback strategies | LiveKit Documentation, with interactive elements
2 Views
Open in Slack
PreviousNext
Powered by