Hello! I'm loving the product and have started testing it with my models, however I'm facing an issue that is preventing me from using the redteaming functionality.
I have an Azure OpenAI model hosted there, which I target through a simple HTTP request which just uses a token for authorization, without API key needed. After setting up the target, when I run the 'Test Target' function, I receive a correct response.
However, when I reach the last step and Run the redteaming with the given config, I receive the following error:
------------------------------------------
Using strategies:
Test Generation Summary:
• Total tests: 5
• Plugin tests: 5
• Plugins: 1
• Strategies: 1
• Max concurrency: 1
Extracting system purpose...
Error in extraction: API error: 401 Unauthorized
{"error":{"message":"Incorrect API key provided: **********apikey**********. You can find your API key at
https://platform.openai.com/account/api-keys.","type":"invalid_request_error","param":null,"code":"invalid_api_key"}}
Error running redteam: Error: Failed to perform extraction: API error: 401 Unauthorized
-------------------------------------
This is the configuration .yaml
# yaml-language-server: $schema=
https://promptfoo.dev/config-schema.json
description: Azure Test
targets:
- id: http
config:
url:
https://myurl.azurewebsites.net/endpoint/generate
method: POST
headers:
Cookie: cookieValue
Content-Type: application/json
Authorization: Basic authToken
body: |-
{
"messages": [
{
"role": "user",
"content": "{{prompt}}",
}
]
}
stateful: false
label: AzureOpenAI-Test
prompts:
- '{{prompt}}'
redteam:
purpose: ''
plugins:
- id: cyberseceval # Tests prompt injection attacks from Meta's CyberSecEval dataset
strategies:
- id: prompt-injection # Tests for direct prompt injection vulnerabilities
Thanks in advance!