Hii, love the vision for this product I'm highly enthusiastic to use this solution of promptfoo to have a project application wise analysis + evaluation dashboard along with a quick configurable playground, where everything right from the test iterations, model used, prompt set used, model configuration of the run, golden data run. Etc. all must be persisted and stored in a DB for generating a data point in the main application/project wise configurable dashboard, where the Cost, Accuracy, Precision, Recall, Latency, etc. values across various iterative runs are visualised simply and clearly, effectively allowing for a call for action so as to warrent a prompt update or a model change or something else. FYI: The usecsse is in the toxicity screening domain, i.e. moderation. Kindly share your thoughts and how we can go about this proactively, open to all lets keep this thread alive.

Greetings! That does sound like something you could build. Promptfoo is open source, so we encourage folks to build on it!

What is the support that could be provided from promptfoo? Does this idea makes sense in the vision of the framework?