https://promptfoo.dev/ logo
Best practices for metrics and diffs?
# ❓|questions
y
How to best track metrics, version data alongside code, etc? do ppl log output to a different db for analysis?
i
I've seen some people use the JSON outputs (
promptfoo eval -o output.json
) to do their own custom analysis/tracking... however it's much more common for people to use the web ui (
promptfoo view
)
y
are there examples of what the custom analysis might look like?
the webui is a great starting point
i
hmm, I don't have any examples to point to. Mostly I think companies do it in-house. The output file types are here: - in case it helps you parse