https://promptfoo.dev/ logo
Grader Endpoint
# questions
u
I’m currently running a red team configuration and I want to use remote generation from Promptfoo, but force grading to use my own Azure OpenAI endpoint. I have added the defaultTest override as can be seen in the image. When I set PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=true, I can see my Azure endpoint being hit in the logs for grading. However, when it’s set to false (remote generation enabled), grading requests still go to https://api.promptfoo.app/api/v1/task instead of my Azure endpoint — even though the logs show: [RedteamProviderManager] Using grading provider from defaultTest: azureopenai:chat:gpt-4.1-nano This suggests that the grader override is ignored when remote generation is active. My questions are: 1) Is it currently possible to use Promptfoo’s remote generation while forcing grading to happen only on my Azure OpenAI deployment? 2) If so, what’s the correct configuration to achieve this? 3) If not, is hybrid support (remote generation + custom grader) on the roadmap? https://cdn.discordapp.com/attachments/1423413621488095416/1423413621672902736/image.png?ex=68e038bd&is=68dee73d&hm=39161e9a3001840e6ba6a65f4d3a209333aaa41675d5752700844db3c01983f9&