https://promptfoo.dev/ logo
Does caching work for python LLM provider?
# ❓|questions
p
Hello, I just configured my python provider for Claude 2.1 (I think it is not supported currently) and the caching does not seems to work.
i
Are you using the
exec
or
python
provider type?
Actually, it seems neither of those support caching right now. This would not be too difficult to add though
p
python provider, it would be great to add it because now in order to compare several prompts I have to run them all again and again which in my case is thousands of tokens
i
Just created a PR to enable caching for these providers: https://github.com/promptfoo/promptfoo/pull/435
it will be in the next release 🙂
s
I'm looking for the same thing in the custom API provider. It doesn't look like the caching implementation is public for me to build within my own provider. Is something like this on the horizon for custom API providers?