https://promptfoo.dev/ logo
Testing Mistral-Ollama via Open AI API (openwebUI)
# ❓|questions
s
I am trying to perform a red team evaluation against mistral running locally on Ollama. Since i have Open Web UI installed i am utilizing the Open AI API provided by Open Web UI. I am able to successfully make a curl request through this API. However , while setting up a custom target with attached YAML, it gives me model not found error. Curl request, YAML config and Promptfoo error attached.. https://cdn.discordapp.com/attachments/1358033855214518384/1358033855457661178/image.png?ex=67f25f16&is=67f10d96&hm=05041d85b7826735091f294b4239565996cba634ad666d3f68c53b2f1c92e83a& https://cdn.discordapp.com/attachments/1358033855214518384/1358033855986008265/image.png?ex=67f25f16&is=67f10d96&hm=6dacc8d053d81540bb4c8a47c3df29f5b1c56df2871e62239b7721d0c43e1ab5& https://cdn.discordapp.com/attachments/1358033855214518384/1358033856321683486/image.png?ex=67f25f16&is=67f10d96&hm=6a25c2777fbf936c588faf5c44f504e0ab120dd752d3f5e6832b2ee89f38acab&
Note: I was able to successfully run redteam on chatgpt4o using promptfoo....
s
Can anyone please help me with this?
i
your config looks strange. it should just be
Copy code
yaml
targets:
  - id: openai:chat:mistral:latest
    config:
      apiBaseUrl: http://localhost...
      temperature: ...
not the nested
providers
inside
targets.config
i
The base url should just be http://localhost:3000/api I believe
s
I made the changes and not getting any error now...however i want to ensure my understanding of "promptfoo red team" command is correct. My promptfooconfig.yaml config is as mentioned below. The apiKey set here is generated from Openweb UI which is locally hosting mistral. targets: - id: openai:chat:mistral:latest config: apiBaseUrl: http://localhost:3000/api apiKey: 'sk-3728ca66f0dd4425bfeaa3-------' temperature: 0.5 In environment variable i have set OPENAI_API_KEY to the paid API key from OpenAI web console. Based on attached "promptfoo redteam run --verbose" output i can understand... 1. On running this command it first generates requests towards api.promptfoo.org to generate harmful prompts. 2. It then sends these prompts to locally hosted mistral via API hosted on OpenwebUI. 3. The response received from above prompts are graded. For grading the prompts are sent to gpt40. For grading to work an active OpenAI API key is required and set in environment variables. @ian Would request you to go through attached output and confirm if i was actually able to test the locally hosted mistral using promptfoo and my above understanding of "promptfoo redteam run" is correct. https://cdn.discordapp.com/attachments/1358033855214518384/1359079245380587601/Verbose_output.txt?ex=67f62cae&is=67f4db2e&hm=b2dfde8626103cb76610baad44bb4524a45a675a40086328923b050dee946bf0&