https://retellai.com/?utm_source=discord&utm_medium=seo logo
Powered by
Possible to setup a secondary LLM if the primary m...
# ❓・ai-evy-support
c
@AI Evy Possible to setup a secondary LLM if the primary model goes down for some reason?
a
Thanks for tagging me @Codyx. I'll look through relevant sources and get back to you shortly.
### Answer Yes, Retell AI has a built-in Seamless Fallback system that automatically handles LLM failovers. [(1) ]() How LLM Fallback Works: If your call is powered by an OpenAI LLM and OpenAI becomes temporarily unavailable, Retell AI will automatically route that conversation through a hosted backup model (e.g., Azure-hosted equivalent) to maintain conversational flow. Key Benefits: - Reduced risk of failed calls—calls stay connected even if a provider goes offline - No disruption to customer experience—callers remain unaware of any backend transition - Greater operational resilience—built-in redundancy ensures voice operations continue normally Configuration: You can configure your fallback preferences in the Security & Fallback Settings Menu in the dashboard. Additionally, if you want to enable a more stable connection when using OpenAI LLM, you can toggle the High Priority Connection option (note: this comes at double the cost due to dedicated resources) .
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
​ To learn more, see the sources I considered: [[1] How Retell AI Handles Outages with a 99.99% Uptime | Retell AI](https://www.retellai.com/blog/how-retell-ai-handles-outages-with-a-99-99-uptime) ​
c
The only option I see, which I have toggled already, is the 'Fallback Voice ID'- I assume that is a speech engine, not an LLM. Am I wrong? I use Gemini as the agent's primary model, with Openai-cimo as the fallback voice ID. Does that mean if Gemini goes down, OpenAI takes over?
@AI Evy The only option I see, which I have toggled already, is the 'Fallback Voice ID'- I assume that is a speech engine, not an LLM. Am I wrong? I use Gemini as the agent's primary model, with Openai-cimo as the fallback voice ID. Does that mean if Gemini goes down, OpenAI takes over?
a
### Answer You're correct—**Fallback Voice IDs** are specifically for TTS (Text-to-Speech) provider outages, not LLM failovers. [(1) ]() From the documentation: > "When TTS provider for the selected voice is experiencing outages, we would use fallback voices listed here for the agent. Voice id and the fallback voice ids must be from different TTS providers." So your 
openai-cimo
fallback only kicks in if your primary voice's TTS provider (like ElevenLabs or Deepgram) goes down—it does not provide LLM redundancy if Gemini becomes unavailable. Regarding LLM fallback: The API documentation I have access to does not show a configurable secondary LLM option at the agent level. The 
response_engine
configuration only allows attaching a single LLM. [(1) ]() [(2) ]() For LLM-level failover configuration, I'd recommend checking the dashboard's Security & Fallback Settings as mentioned in the earlier thread, or reaching out via: - [Contact Sales]() - [Retell AI GitHub]() Would you like me to try answering more generally based on my knowledge?
​ To learn more, see the sources I considered: [[1] Update Voice Agent](https://docs.retellai.com/api-references/update-agent) [[2] Create Voice Agent](https://docs.retellai.com/api-references/create-agent) ​
Thanks for the follow-up, @Codyx. I'll get back to you soon.
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
PreviousNext
Powered by