https://promptfoo.dev/ logo
i
This page should work fine in dev mode if you've run a redteam-generated eval
k
I have already done running redteam but still not showing. Maybe there's a config? Because this is what I've seen in the code: I don't know where to set that config.metadata.redteam. This will actually show a button to display the reports. https://cdn.discordapp.com/attachments/1255776279219671081/1255832488069300284/image_2.png?ex=667e90a5&is=667d3f25&hm=838b2736f2af04c73e698823ee1867686fe70a3de302a7f48e40420c1841490a&
i have also tried and generated the promptfooconfig.yaml as a new project but it only shows the graphs, not the Risk reports.
the metadata should be set automatically
on the eval
k
no, its not showing for me and my friend also. As a workaround, we change the config from EvaluationResult db, by adding metadata: { redteam: true }. To show "Vulnerability Report". Is there a way to set the redteam automatically?
also i am getting this error, even after providing the api key Error: OpenAI API key is not set. Set the OPENAI_API_KEY environment variable or add
apiKey
to the provider config. Error: OpenAI API key is not set. Set the OPENAI_API_KEY environment variable or add
apiKey
to the provider config. at OpenAiChatCompletionProvider.callApi (webpack-internal:///(rsc)/../../providers/openai.ts:323:19) at matchesLlmRubric (webpack-internal:///(rsc)/../../matchers.ts:230:38) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async runAssertion (webpack-internal:///(rsc)/../../assertions.ts:694:16) at async eval (webpack-internal:///(rsc)/../../assertions.ts:143:24) https://cdn.discordapp.com/attachments/1255776279219671081/1255932231738855484/Screenshot_2024-06-27_223457.png?ex=667eed8a&is=667d9c0a&hm=08f4e86b1f922f08d6a67d44be98c23f0e437c8efcff200ec14c1d3d9824e5ba&
i
Just double checking, did you generate the testset using
promptfoo generate redteam
?
By default, the redteam will use OpenAI for attack generation and model grading, which requires an
OPENAI_API_KEY
environment variable. This can be overridden here , but we find that most other models do not work well for this use case
k
put the open ai key in env and its working for now, but whenever i am clicking on view logs in the risk report the open api error returns
also a big thankyou from me and my friend Ed for helping us out. Means a lot.
3 Views