Hello, I just configured my python provider for Claude 2.1 (I think it is not supported currently) and the caching does not seems to work.
i
ian
01/30/2024, 3:51 PM
Are you using the
exec
or
python
provider type?
ian
01/30/2024, 3:52 PM
Actually, it seems neither of those support caching right now. This would not be too difficult to add though
p
paluchnuggets
01/30/2024, 3:57 PM
python provider, it would be great to add it because now in order to compare several prompts I have to run them all again and again which in my case is thousands of tokens
I'm looking for the same thing in the custom API provider. It doesn't look like the caching implementation is public for me to build within my own provider. Is something like this on the horizon for custom API providers?