https://promptfoo.dev/ logo
harmful:hate plugin requires remote generation to ...
# ❓|questions
d
i set this variable : PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=True However i got this message Generating tests for harmful:hate... An unexpected error occurred: harmful:hate plugin requires remote generation to be enabled
m
A number of the harmful plugins, including harmful:hate, rely on a fine-tuned model we provide. This is because generating high-quality attacks for these cases is not feasible with an aligned model (e.g., OpenAI models tend to strongly avoid generating harmful or self-harm, etc. related outputs). This fine-tuned model is hosted remotely and made freely available for Promptfoo users. Unfortunately, if remote generation is disabled (PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=True), plugins relying on this model, such as harmful:hate, won’t function. I’ll update the documentation to better outline which plugins require remote generation. Let us know if you have further questions!
d
can't we use custom models instead of promptfoo models ? Because this is what i thought when i read the docs : > By default, Promptfoo uses your local OpenAI key to perform attack generation. If you do not have a key, it will use our API to generate initial inputs, but the actual attacks are still performed locally. > > You can force 100% local generation by setting the PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION > environment variable to true. Note that the quality of local generation depends greatly on the model that you configure, and is generally low for most models.
Providers The redteam.provider field allows you to specify a provider configuration for the "attacker" model, i.e. the model that generates adversarial inputs.
m
Right now, you can point the PROMPTFOO_REMOTE_GENERATION_URL to whatever you like. I'll think about how feasbile it is to override that specifically. The endpoint is actually a few different models and has lots of business logic built in so it's not as simple as pointing directly to a new model.
redteam.provider also applies to the model used when running attacks in strategies. I'll update the docs to disambiguate this more clearly
If you have GPUs and would like to self host our model we can ship it to you as part of a support contract or we can guarantee 0 day retention in writing etc.
d
i didn't understand correctly what does redteam.provider do with PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION+True 😂 😅
m
Another way of thinking about the models for redteam generation is there are actually three - the unaligned model we use - the aligned model we use to generate all of the open source attacks and use in strategies like iterative - the target model you are attacking
d
doesn't redteam.provider replaces your model if set ?
m
redteam.provider replaces the aligned model
d
and the unaligned model please ?
m
the unaligned model is the remote model
d
this is the doc that i wrote for the redteam feature, i was missing as you can see the unaligned model because i thought that it was the same as the attacker model
m
We can definitely do a better job of making this clear in our docs too
d
but i still don't understand, you have two models that generate adversarial content ?
You actually did a pretty amazing job with the tool and the doc, i understand that it is really complicated having an out of the box view when you are one of the maintainers of the tool but yeah, it can get better of course
m
Yes there are two models - the unaligned model that promptfoo hosts. It's defined here: https://github.com/promptfoo/promptfoo/blob/main/src/providers/promptfoo.ts#L24 - the aligned model that you can override defined here: https://github.com/promptfoo/promptfoo/blob/main/src/redteam/providers/shared.ts#L14 Most plugins exlcuding harmful: use this model and have completely open source definitions. All strategies use this model. And then the target model which is whatever you are attacking.
d
okey, to summarize i can replace every model in promptfoo except the unaligned one that you are using in some plugins like harmful content
the only question i still have is : what is the differnce between the unaigned and aligned model since both generate stuff
why for example can't we use an uncensored model to generate hamrful content instead of the unaligned one ?
i know for sure there is a reason since you provide the possibility to override all the models except the unligned one as you mentioned before
> I'll think about how feasbile it is to override that specifically.
m
uncensored and unalgined are exchangable in this context. You could fine tune a model to remove refusals and then fine tune again to generate harmful content
d
exactly, then why having this : > The endpoint is actually a few different models and has lots of business logic built in so it's not as simple as pointing directly to a new model.
maybe a solution would be : if we override the model, it will simply just call a model if we don't, it will use your endpoint end everything
with this, it would just be a condition, if PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=True, just call a provider or somethig like that instead of raising that the plugin doesn't support this yey if False, you keep your logic
i don't know if it is feasable of course, i'm just trying to give feedback to improve the product
😅
m
> then why having this : We simplified the interface as much as possible in the promptfoo completion provider and have more in the server as a result. I'll think about how to best do this! Thank you so much for your feedback. If you are working for a company that might be interested in hosting promptfoo in a way where absolutely 0 data leaves your system feel free to put time on our calendars.
d
My company is planning to do so in the next year after defining the new budget. That is why i'm heavily testing this tool now haha. And also for my personal project. Of course we are clearly planning to support such a useful tool as yours and hosting it in our servers! thank you again for your help
m
Awesome! Put time on our calendar: https://cal.com/team/promptfoo/solutions and we can give you a demo of enterprise features
2 Views