https://promptfoo.dev/ logo
Troubleshoot asserts context-faithfulness
# ❓|questions
e
Hi team, Is there a way to troubleshoot/debug errors from assert context-faithfulness ? I had one of these notification after running eval with verbose, At least one evaluation failure occurred. This might be caused by the underlying call to the provider, or a test failure. Context: [{"id":"8868fc514e5f7bd1a350480c5a825cd0f5b47b6c0c8da0ca62be71158f638c23","raw":"{{query}}","label":"{{query}}","provider":"http","metrics":{"score":0.6,"testPassCount":0,"testFailCount":1,"testErrorCount":0,"assertPassCount":0,"assertFailCount":1,"totalLatencyMs":8228,"tokenUsage":{"prompt":0,"completion":0,"cached":0,"total":0,"numRequests":1,"completionDetails":{"reasoning":0,"acceptedPrediction":0,"rejectedPrediction":0,"cacheReadInputTokens":0,"cacheCreationInputTokens":0},"assertions":{"total":4876,"prompt":3851,"completion":1025,"cached":0,"numRequests":0,"completionDetails":{"reasoning":0,"acceptedPrediction":0,"rejectedPrediction":0,"cacheReadInputTokens":0,"cacheCreationInputTokens":0}}},"namedScores":{"Faithfulness_Check":0.6},"namedScoresCount":{"Faithfulness_Check":1},"namedScoreWeights":{"Faithfulness_Check":1},"cost":0}}] Thanks
2 Views