Powered by
Enabled caching sonnet, boundary studio shows only...
# help
c
crossentropy
08/13/2025, 6:12 PM
I enabled prompt caching using a custom client on a prompt that is ~7000 tokens static, ~1000 dynamic. Always around ~100-200 output tokens. I'm sure the 4 tokens is wrong. These counts are fine with streaming without caching enabled. This is a streaming call, maybe that could have something to do with it?
https://cdn.discordapp.com/attachments/1405252740741075095/1405252740988534844/image.png?ex=689e2719&is=689cd599&hm=4199b4cfb719736798ffb1ee907b2da48dbfdbca78f14d8e2a4ef724eaba4ddc&
a
aaronv (BAML)
08/13/2025, 6:13 PM
yes we probably dont count these correctly in the studio. We'll verify in v2 of the studio we're aiming to launch
c
crossentropy
08/13/2025, 6:47 PM
Okie
Previous
Next