https://promptfoo.dev/ logo
Scaling self-hosted Promptfoo in Kubernetes (K8s)
# ❓|questions
y
Hi everyone, We're currently experimenting with self-hosting Promptfoo in a Kubernetes (K8s) environment. However, we're facing issues when trying to scale the deployment to more than one replica. The problem arises because the /api/job/:id endpoint and a few others do not function correctly in a multi-replica setup. It seems that running jobs are stored in memory using: export const evalJobs = new Map(); This makes it instance-specific, so other replicas don't have access to the job state. Is it possible to scale Promptfoo horizontally in Kubernetes? If so, what strategies or workarounds would you recommend to support multiple replicas? Any advice or examples would be greatly appreciated! Thanks 🙂
w
Hi @yurii.izo_09523, we don't currently support horizontally scaling self-hosted Promptfoo on K8s. Can you tell me a bit more about your use case? Perhaps there is something else which could work for you
i
our enterprise version does handle horizontal scaling if that's something you'd be interested in. happy to discuss further
y
got it, thanks for the clarification 🙂
2 Views