https://promptfoo.dev/ logo
Is there a way to store cache in the cloud?
# ❓|questions
t
Hi, we plan to use promptfoo in my team, we aim to run it both locally and in the CI. Our prompts are expensive, and we aim to run hundreds of them. We would like to have a shared cache store to all the developers, so once I run a prompt locally on my machine, it gets cached, and another developer will use my cached results will will run it on their local machine. Is there a way to make it using promptfoo?
o
@Treffie https://www.promptfoo.dev/docs/usage/self-hosting/ You can self-host an eval instance, then make sure that everyone is running their evals from the centralized machine rather than on their own machines. There isn't an easy way to have everyone using the same cache out of the box. Another idea is to set your cache to a shared network folder, but if two people start running evaluations at the same time.... Not quite sure what would happen.
i
+1 to onawa's answer. Right now self hosting or shared network folder is the best way to do this Alternatively, you could set up a cache proxy and use a custom provider to pass through the proxy. That way you have full control over the cache policy etc
o
Ooooh, yeah, that's a good idea as well. Set up a LiteLLM instance, then use their caching function and then people could still run from their local machines without having to change their cache location. https://docs.litellm.ai/docs/proxy/caching
Plus, all the added benefits of running the proxy itself including standardizing all your calls to an OpenAI API, budgets, and API key management so you only have to give each developer a single key and not the service API keys in case they push them to git or something like that.