https://promptfoo.dev/ logo
What data is sent to PromptFoo's API during remote...
# ❓|questions
j
I understand that PromptFoo uses their own fine-tuned models for attack generation, which are accessible only through PromptFoo's API. Some plugins (like harmful) requires this models even when setting PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=true. For these cases, I assume that the whole config file is sent to PromptFoo's server, exposing, for example, the "purpose" section. My question is then particularly about the jailbreak strategy. On the documentation jailbreak is described as following: 1) Starting with a base prompt that attempts to elicit undesired behavior 2) Using an LLM-as-a-Judge to: 2.1) Analyze the AI's response 2.2) Track the conversation history 2.3) Generate increasingly refined prompts based on previous attempts 4) Repeating this process for a configurable number of iterations 5) Selecting the most effective prompt variation discovered Is the generation of point 2.3) taking place again in PromptFoo's servers? I.e., are all the answers of my LLM being "exposed"? Also is it possible to completely replace PromptFoo's unaligned model for a local unaligned one? Thank you
i
Hi John, sorry for the delay here. > For these cases, I assume that the whole config file is sent to PromptFoo's server, exposing, for example, the "purpose" section. This is not quite true. Only the Purpose section and the selected plugins are sent. Details such as the provider itself are not sent. > Is the generation of point 2.3) taking place again in PromptFoo's servers? By default, without a source of inference, the Promptfoo client will use the server. If you [override the redteam provider](https://www.promptfoo.dev/docs/red-team/configuration/#changing-the-model) OR set your own
OPENAI_API_KEY
environment variable, then your LLM's answers are never exposed to the Promptfoo server. > Also is it possible to completely replace PromptFoo's unaligned model for a local unaligned one? This is not well supported at the moment because we've found there are big difference in quality/capability among unaligned models. On the enterprise side we do have the ability to self-host the models if that's something that interests you. Hope this helps and let me know if I can clarify anything.
j
Hi Ian, thank you for the answer, really appreciated. If I can, I'd like to ask one last clarification about this: > If you override the redteam provider OR set your own OPENAI_API_KEY environment variable, then your LLM's answers are never exposed to the Promptfoo server. This is true only for the plugins which do not require remote generations, right? For example the "harmful" plugins are not supposed to work when using PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=true, even when overriding the redteam provider. Am I correct? Thank you again, Keep up the good work!
i
Yes, that's correct.
PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=true
will disable these, so you can ensure that your LLM's answers never leak, but yeah you'd have to omit those plugins
j
Thanks again! That answers all my doubts.