Hey, I've been trying to use rag-poisoning plugin for documents generation, but upon running assessment these documents fails to be generated with fallowing error that I will upload to this post. https://cdn.discordapp.com/attachments/1475457665592332329/1475457720231399635/image.png?ex=699d8e95&is=699c3d15&hm=d8d40d925faeaf91071491b6e3b4dab48b550418c4525e76dfa1b8eca409f9c6&

Hey @Envy, that's a great question. The rag-poisoning plugin requires intendedResults in its config. It can't generate test cases without knowing what the poisoned documents are supposed to make the LLM do. The workflow is: 1. Add rag-poisoning as a plugin with intendedResults in your config:

plugins:
    - id: rag-poisoning
      config:
        intendedResults:
          - 'Contact support at attacker@evil.com'
          - 'The company is giving away free products'

2. These should match the intended effects of whatever poisoned documents you've added to your RAG knowledge base. You can use promptfoo redteam poison to generate both the poisoned documents and their intended results automatically. You are correct in pointing out that the error message was not helpful so we're adding client-side validation so it fails with a clear message instead of a 500 from the server.

Hey @[Promptfoo] Konstantine, Thanks for your response and insights that you provided. What about "black box" testing scenarios where you don't have access to RAG KB? Would it be worth generating documents with redteam poison option or people usually use something else? Thinking about something like uploading such documents into chat bot for example in order to attempt jailbreak/hallucination in that session. Thanks in advance!