https://livekit.io logo
Join Slack
Powered by
Why doesn't the ApiConnectionError fallback to the...
# ask-ai
r
Why doesn't the ApiConnectionError fallback to the next llm in my fallback adapter
Copy code
livekit.agents._exceptions.APIConnectionError: Connection error. (body=None, retryable=False)
2025-07-28 20:28:53,120 - livekit.agents - ERROR - livekit.plugins.openai.llm.LLM failed after sending chunk, skip retrying
2025-07-28 20:28:53,117 - livekit.agents - WARNING - livekit.plugins.openai.llm.LLM failed, switching to next LLM
t
Thanks for reaching out, John. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
The fallback adapter is designed to handle 
APIConnectionError
and switch to the next available LLM instance in most cases. However, there is an important exception where fallback is prevented: • If chunks have already been sent to the client (indicated by 
chunk_sent
), the adapter will not attempt fallback. This is to avoid sending inconsistent or duplicate responses. In such cases, the error is re-raised immediately, and fallback is skipped. From the logs you provided, it seems the error occurred after a chunk was already sent, which is why the fallback mechanism did not proceed to the next LLM. This behavior is intentional to maintain response consistency. If you need further clarification or assistance, let me know. Sources: livekit-agents/livekit/agents/llm/llm.py
r
What's the best way to handle the errors, I want to fallback
its crashing my application
t
Hey there! Let's talk about keeping your application running smoothly and handling potential hiccups like a pro. I've got some friendly tips to help you prevent crashes and make your system super resilient. • Configuration Best Practices: Think of 
max_retry=0
as your first line of defense. It's like telling your API connection, "Let's not keep trying if something's not working." Adjust those fallback settings like 
attempt_timeout
, 
max_retry_per_llm
, and 
retry_interval
to fit your specific needs. • Error Handling Strategy: Imagine you're a friendly navigator guiding your system through rough waters. Gracefully move through fallback instances, marking the ones that aren't cooperating. Only raise that 
APIConnectionError
when you've exhausted all options. • Recovery Mechanisms: Set up some smart background tasks that can automatically bring failed instances back to life. Just make sure you're not accidentally running multiple recovery missions at the same time! • Logging and Monitoring: Keep a friendly log of what's happening – a warning here, an error there. It's like leaving breadcrumbs for yourself to understand what's going on behind the scenes. • Streaming Considerations: Be careful about partial data. You want to make sure you're not sending mixed or incomplete messages. Think of it like making sure you finish your sentence before switching topics. Example Usage:
Copy code
from livekit.agents import llm
from livekit.plugins import openai, groq

llm_adapter = llm.FallbackAdapter([
    openai.LLM(),
    groq.LLM(),
])
These tips will help you create a super robust and friendly application. Need any more help? I'm always here to chat and guide you through! Sources: livekit-plugins/livekit-plugins-tavus/livekit/plugins/tavus/api.py | Events and error handling | LiveKit Docs
r
why doesn't it recover? How can i tell the fallback adapter to have max_retry=0 on specific errors?
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
Open in Slack
PreviousNext
Powered by