Accessing cache_read_input_tokens and cache_creati...
# helpa
Anders
12/07/2025, 10:53 PMI see in the [docs](https://docs.boundaryml.com/guide/baml-advanced/collector-track-tokens#best-practices) that it mentions HTTP response will be null for streaming responses, but normally this is how I would access cache_read_input_tokens and cache_creation_input_tokens for Claude models, so how am I supposed to access it for a streamed response? Collector usage is available, but I need the input token cache read/write info.
Using Bedrock provider, Anthropic provider as fallback.
Anders
12/07/2025, 11:25 PMI notice
collector.usage.cachedInputTokensnullb
baml-sam
12/08/2025, 12:08 AMHm... I'm not sure if we expose this through the collector today.
baml-sam
12/08/2025, 12:10 AMDigging into the code
baml-sam
12/08/2025, 12:30 AMAh, we never documented this... that's our mistake
baml-sam
12/08/2025, 12:31 AMYou can access the SSE responses via
call.sse_responses()baml-sam
12/08/2025, 12:32 AMI've also gone ahead and filed https://github.com/BoundaryML/baml/issues/2780 for this
a
aaronv (BAML)
12/08/2025, 10:47 PMwe likely missed adding this on the Anthropic streaming API, will see if i can get it patched next release
a
Anders
12/09/2025, 12:32 AMAppreciate it. I believe I checked the SSE responses when I was trying to find a workaround and I think it was empty?
a
aaronv (BAML)
12/09/2025, 6:16 PMyeah we may not have populated the collector for Anthropic yet on streams, ty for the report
aaronv (BAML)
12/12/2025, 8:06 PMok i think i found the bug here
aaronv (BAML)
12/12/2025, 8:27 PMi confirmed it does cache with streams in our test:
https://cdn.discordapp.com/attachments/1447360451041300621/1449135483899871323/image.png?ex=693dcc16&is=693c7a96&hm=84172a43e148f3531277f0e198c90f65d2d24b0d780de6c7cb225015b80a2d4f&
aaronv (BAML)
12/12/2025, 8:27 PMand now we have sse responses
aaronv (BAML)
12/12/2025, 8:27 PMwait, is this anthropic on vertex, or anthropic raw api? @Anders
aaronv (BAML)
12/12/2025, 8:55 PMfix will go out 0.215.0 next week
a
Anders
12/13/2025, 12:18 AMThis was anthropic anthropic and bedrock anthropic