https://promptfoo.dev/ logo
Context faithfulness results differs much between ...
# ❓|questions
w
Hi, I've run promptfoo twice, both times on same config.yaml and on first run I've got context faithfulness equal to 1, but second run gave me 0. I'm using Claude V2 as default model grader. I know that those metrics aren't deterministic, but such difference (from 100% to 0%) is quite strange. And even response generated by LLM didn't change. Did anyone faced such issue?
v
how did you use context-faithfulness? are you using via tests: test.csv format way?
2 Views