the fallbackadaptor only works if there is api sid...
# ask-aia
adorable-bear-14321
04/15/2026, 9:44 AMthe fallbackadaptor only works if there is api side error
if there is client side error like invalid syntax etc. the fallback adaptor doen;t switch to the next stt
is this the expected behaviour?
✅ 1
t
tall-airline-16756
04/15/2026, 9:44 AMThanks for reaching out, Harshita. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
04/15/2026, 9:44 AMThanks — I’m responding about whether the fallback adaptor should switch providers on client-side errors (like invalid syntax) versus API-side errors.
Short answer: That behavior is expected. The FallbackAdapter is designed to trigger when a provider fails (API-level errors, timeouts, outages). It does not automatically treat client-side issues (invalid input/syntax, malformed audio, or similar) as provider failures, so it won’t switch for those cases.
If you’d like the adaptor to try another STT on client-side failures, I can suggest approaches:
• Detect specific client-side error conditions in your pipeline and explicitly call the fallback provider when they occur.
• Add validation/normalization before STT so fewer client errors reach the provider.
• Log examples of the client errors so we can decide which errors should trigger an automatic fallback.
Which approach would you prefer, or can you share a sample client error/stack trace so I can recommend exact changes?
Sources: Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Sequential Pipeline Architecture for Voice Agents | LiveKit | Sequential Pipeline Architecture for Voice Agents | LiveKit, with interactive elements
a
adorable-bear-14321
04/15/2026, 9:48 AMi have two stt
main stt = sarvam
fallback stt = azure
this is how im returning the stt
Copy code
def create_stt(self, agent_info: AgentInfo):
return stt.FallbackAdapter([
_build_sarvam_stt_instance(self, agent_info),
_build_azure_stt_instance(self, agent_info),
]) Copy code
def _build_sarvam_stt_instance(provider: STTProvider, agent_info: AgentInfo):
stt_details: STTDetails = agent_info.config.stt_details
return sarvam.STT(
model=agent_info.model_values.stt.model or provider.get_config_value('sarvam', 'model', ),
language=stt_details.language or provider.get_config_value('sarvam', 'language', ),
mode=stt_details.mode,
prompt=stt_details.prompt or provider.get_config_value('sarvam', 'prompt', ),
high_vad_sensitivity=stt_details.high_vad_sensitivity or provider.get_config_value(
'sarvam', 'high_vad_sensitivity', ),
sample_rate=stt_details.sample_rate or provider.get_config_value('sarvam', 'sample_rate', 'int'),
flush_signal=stt_details.flush_signal or provider.get_config_value('sarvam', 'flush_signal', ),
input_audio_codec=stt_details.input_audio_codec or provider.get_config_value('sarvam', 'input_audio_codec', ),
) Copy code
def _build_azure_stt_instance(provider: STTProvider, agent_info: AgentInfo) -> azure_stt_1_3x.STT:
stt_details: STTDetails = agent_info.config.stt_details
profanity_option = AzureSTTProvider.get_profanity_option(
stt_details.profanity or provider.get_config_value('azure', 'profanity', ))
languages = stt_details.languages
if languages is None:
languages = []
single_language = stt_details.language
combined_languages = [
lang.strip()
for lang in (languages + ([single_language] if single_language else []))
if lang and lang.strip()
]
return azure_stt_1_3x.STT(
speech_key=stt_details.speech_key or provider.get_config_value('azure', 'speech_key_env', 'env'),
speech_region=stt_details.speech_region or provider.get_config_value('azure', 'speech_region_env', 'env'),
speech_auth_token=stt_details.speech_auth_token or provider.get_config_value('azure', 'speech_auth_token', ),
sample_rate=stt_details.sample_rate or provider.get_config_value('azure', 'sample_rate', 'int'),
num_channels=stt_details.num_channels or provider.get_config_value('azure', 'num_channels', 'int'),
segmentation_silence_timeout_ms=stt_details.segmentation_silence_timeout_ms or provider.get_config_value(
'azure', 'segmentation_silence_timeout_ms', 'int'),
segmentation_max_time_ms=stt_details.segmentation_max_time_ms or provider.get_config_value(
'azure', 'segmentation_max_time_ms', 'int'),
segmentation_strategy=stt_details.segmentation_strategy or provider.get_config_value(
'azure', 'segmentation_strategy', ),
language=combined_languages,
profanity=profanity_option,
)t
tall-airline-16756
04/15/2026, 9:49 AMThanks — I’m responding about why your FallbackAdapter doesn’t switch to Azure when Sarvam raises a client-side/configuration error (language="hiiii") and the session terminates.
Short answer: That’s expected. FallbackAdapter only triggers for provider-level failures during streaming/processing (API errors, timeouts, outages). It does not intercept initialization/configuration exceptions raised before the provider starts, so an invalid parameter in Sarvam causes an exception that prevents the session from initializing and therefore no automatic fallback occurs.
What you can do (options):
• Validate before building the STT instances: check/normalize
languagesessionNonesession2 Views