Hi everyone,
We're currently experimenting with self-hosting Promptfoo in a Kubernetes (K8s) environment. However, we're facing issues when trying to scale the deployment to more than one replica.
The problem arises because the /api/job/:id endpoint and a few others do not function correctly in a multi-replica setup.
It seems that running jobs are stored in memory using:
export const evalJobs = new Map
();
This makes it instance-specific, so other replicas don't have access to the job state.
Is it possible to scale Promptfoo horizontally in Kubernetes?
If so, what strategies or workarounds would you recommend to support multiple replicas?
Any advice or examples would be greatly appreciated! Thanks 🙂