https://www.boundaryml.com logo
Enabled caching sonnet, boundary studio shows only...
# help
c
I enabled prompt caching using a custom client on a prompt that is ~7000 tokens static, ~1000 dynamic. Always around ~100-200 output tokens. I'm sure the 4 tokens is wrong. These counts are fine with streaming without caching enabled. This is a streaming call, maybe that could have something to do with it? https://cdn.discordapp.com/attachments/1405252740741075095/1405252740988534844/image.png?ex=689e2719&is=689cd599&hm=4199b4cfb719736798ffb1ee907b2da48dbfdbca78f14d8e2a4ef724eaba4ddc&
a
yes we probably dont count these correctly in the studio. We'll verify in v2 of the studio we're aiming to launch
c
Okie