since upgrading to 217 I am getting this error a lot in longer running functions : rror: BamlValidationError(message=LLM Failure: reqwest::Error { kind: Request, url: "xxxxxublishers/google/models/gemini-3-flash-preview:generateContent", source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(Io, Custom { kind: BrokenPipe, error: "stream closed because of a broken pipe" })) } (Unspecified error code: 2) - reqwest::Error { kind: Request, url: "xxxxgemini-3-flash-preview:generateContent", source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(Io, Custom { kind: BrokenPipe, error: "stream closed because of a broken pipe" })) }, raw_output=, prompt=user: Review the list of invoices against the provided document text.. any tricss tips for this ?

What were you using before? 0.215?

214

Are you streaming them?

no streaming

Do you have a small repro whereit happens often?

sorry I am using the async function await in python

You have to import the async client if you want the async one

i am already doing that actually from lib.baml_client.baml_client.async_client import b

If you do have the full baml file and python file that repros it i can take a look

this is private work code I can't share

How often does it happen? 100% of the time?

50% of the time now

And downgrading fixes it? Foes the output type contain any unions?

out put no union at all

When using version 0.217, I also encountered this issue once before, with the same error message. However, I always thought it was a network problem, and it hasn't occurred a second time so far.

actually the version I was on that was stable was 210

Ill see if i can repro this friday

now with version 210 the complex workflow is workging as before

How long do your requests to gemini last?

between 10 and 25 min

ok I think i know what the issue is. We introduced timeouts, and the defaults are set to 5min timeout. If every request of yours takes that long it will always timeout. And the other issue is we should give you a 'TimeoutError' but we do not. If you do have time to try 0.218.0 at some point, with this configuration, lmk how it goes! It sets the request timeout to basically 30min

client<llm> MyClient {
  provider openai
  options {
    model "gpt-4"
    api_key env.OPENAI_API_KEY
    http {
      connect_timeout_ms 5000      // 5 seconds to connect
      request_timeout_ms 1800000     // 30 mins total
    }
  }
}

it would be great if the baml docs was updated with this new bahaivor as it still says none

Ah yes we can add our defaults there or perhaps we will just increase our default to 30min

I've been trying to reproduce this, @arafat . Thanks for your help so far! I'm on 0.218.0 and using the python bindings with the same gemini model as you. What I've found so far: - Manually setting a connection timeout of 10 seconds in the BAML client config results in a timeout error (Request timed out) - Cutting my wifi mid-request after 10 seconds results in a timeout error about 30 seconds after the cut (Request timed out) - Running long queries works, when the response takes less than about 3 minutes - I have trouble making a query that takes longer than 2 minutes. When I ask for long lists of things, the model stops itself and asks me to make a followup request for more items. Your pasted client definition uses gemini-3-pro-preview, which is a different model than flash I think. Your error message mentions flash. So I'll rerun my tests with that model too. Ok - same results when using gemini-3-flash-preview. I have not managed to reproduce your issue. Do you have any tips for keeping the model talking for > 3 minutes?

I am passing the model a list of parsed invoices to deduplicate based on the grand total of all the invoices. the list contians 300 invoices with each invoice class model has about 8 fields. this request is what takes the longest. Very nieche use case to be honest with you, I will try to use 218 with the increased timeouts.

Very helpful context! That is probably why the model doesn't intentionally stop for you and ask you to resume with a fresh page - you can't do that for deduplication. I will pick this debugging task back up with a similar query!

Got the same error but my use case takes around 10seconds, and we suddenly got 4min. Looking on Gemini Studio I saw we hit the Rate Limit, but instead of getting Rate limit error, and continuing to the fallback strategy (other model & provider), it waited for the whole timeout. (Same version 217)

^ any token returned by the llm provider in the main content field. Most providers put thinking tokens into a separate field, so those would not count toward the first-token timeout, but for models that put thinking token in the main response (like DeepSeek R1 with its

<thought>

block), thinking tokens would count toward first-token timeout.