https://promptfoo.dev/ logo
Red Team Config Examples
# ❓|questions
g
Hi, Love the idea of promptfoo! I'm trying to set up some simple examples of evaluating promptfoo for my organization. Are there example redteam configs? Like something for ChatGPT, OpenRouter, PortKey.ai, and a local chatbot server? Any tips are much appreciated!
i
check out the examples directory - redteam examples start with
redteam-*
here's a quick one with an openai target: https://github.com/promptfoo/promptfoo/blob/main/examples/redteam-minimal/promptfooconfig.yaml here's an example with a local model: https://github.com/promptfoo/promptfoo/tree/main/examples/redteam-ollama here's an example with a local chatbot: https://github.com/promptfoo/promptfoo/tree/main/examples/redteam-chatbot and agent: https://github.com/promptfoo/promptfoo/tree/main/examples/redteam-agent hope this helps!
g
what's the easiest way to run a redteam test comparing a secure model to an insecure model quickly? should i get an aws box and load the different models with ollama?
i
depends what models you're comparing. Local inference can sometimes be a headache for large models. I really like OpenRouter for this https://www.promptfoo.dev/docs/providers/openrouter/
g
grateful @ian ! i'll give that a try 🙂
2 Views