https://www.boundaryml.com logo
🟡 Question on retry policies and fallback strateg...
# questions
t
Hello Baml team, I had a question regarding retry policies and fall back. I use BAML for structured output in production and would like to improve stability by introducing a retry and fall back strategy. I currently have two clients set up in fallback policy. Here is an example:
Copy code
python
retry_policy FallbackRetryPolicy {
  max_retries 3
  strategy {
    type exponential_backoff
    delay_ms 1000        // initial delay before first retry
    multiplier 2         // each retry waits 2x longer
    max_delay_ms 10000   // cap the delay at 10 seconds
  }
}


client<llm> CLAUDE_SONNET_4_5_GPT_5_4_Fallback {
  provider fallback
  retry_policy FallbackRetryPolicy
  options {
    // This will try the clients in order until one succeeds
    strategy [Claude_Sonnet_4_5, AzureGPT5_4_US_EAST_1]
  }
}
I would like understand the sequence of the try in this case for the above strategy, will the 3 time retry happen for Claude_Sonnet_4_5 first before it falls back to AzureGPT5_4_US_EAST_1 ? Ideally I would like the behaviour to be that a requests are first tried with Claude Sonnet and then GPT-5.4 before any retries are made. To achieve this, do I need to attach the retry policy to the individual clients ? Additionally is there anyway to log when this retry strategy has taken effect or when a fallback has happened ?
a
as i understand it this should try claude then gpt5 each retry attempt
so
[fallback_list] * (1 + max_retries)
you can use the
BAML_LOG=info
environment variable to see logs for retries