Powered by
Tracking tokens per second?
# ❓|questions
w
Will
04/24/2024, 12:18 AM
Would love to get tokens per second listed as a metric for each prompt that is tested. We're testing prompts x models x providers all at once sometimes and it's a nice way to evaluate which provider is giving us the best speeds
i
ian
04/24/2024, 3:50 AM
It's a bit subtle, but for runs that are not cached it will show tok/sec in the header. Is that what you want?
https://cdn.discordapp.com/attachments/1232485675618795643/1232539062771122196/image.png?ex=6629d2f4&is=66288174&hm=664726b7eb97a949702a65660573be9e015c2195c371fd043871416c10a4fe4c&
ian
04/24/2024, 3:50 AM
you can force it to ignore the cache with
promptfoo eval --no-cache
w
Will
04/24/2024, 4:07 AM
ooh yeah this is exactly what i was hoping for
Will
04/24/2024, 4:07 AM
thank you!
Previous
Next