0x64marsh
08/27/2024, 8:48 PMdescription: My first red team
prompts:
- file://./prompt.json
providers:
- id: python:./main_provider_script.py
redteam:
provider:
id: 'python:./redteam_provider_script.py'
plugins:
- competitors
- contracts
- harmful:chemical-biological-weapons
strategies:
- id: base64
- id: crescendo
purpose: >-
this is the llm application I'm testing
entities: &ref_0
- Leo Smith
defaultTest:
metadata:
purpose: >-
this is the llm application I'm testing
entities: *ref_0
options:
provider:
text:
id: python:./moderation_provider_script.py
embedding:
id: python:./moderation_provider_script.pyian
08/27/2024, 10:19 PMian
08/27/2024, 11:39 PM0x64marsh
08/28/2024, 5:41 AMian
08/28/2024, 5:43 AMian
08/28/2024, 5:43 AM0x64marsh
08/28/2024, 5:45 AM0x64marsh
08/28/2024, 9:29 PMdefaultTest->options->provider doesn't appear to be being called.
Instead the RedTeam provider appears to be doing the grading.
Is this intended behaviour?
Extra Info
-----------
attempted the following configs:
promptfoo eval --verbose --grader python:./moderation_provider_script.py -c redteam.yaml
defaultTest:
metadata:
purpose: >-
LLM Application purpose
entities: *ref_0
options:
provider:
text:
id: python:./moderation_provider_script.py
embedding:
id: python:./moderation_provider_script.py
defaultTest:
metadata:
purpose: >-
LLM Application purpose
entities: *ref_0
options:
provider: python:./moderation_provider_script.pyian
08/28/2024, 10:16 PMian
08/28/2024, 10:16 PM0x64marsh
08/29/2024, 12:23 AMian
08/29/2024, 5:15 AM0x64marsh
08/29/2024, 12:14 PMian
08/29/2024, 4:48 PM0x64marsh
09/05/2024, 4:32 AM