Hello @michaelmichaelmichael , it worked only after I set PROMPTFOO_DISABLE_REMOTE_GENERATION=true. Without this environment setting, llm-rubric requests were sent to the remote endpoint even if I had the local LLM under redteam.
I am also a little bit confused with the terms, for example I see "grader" setting in the command line options but in the yaml file it's called "provider".
Or, the environment variable is called PROMPTFOO_DISABLE_REMOTE_GENERATION, but I am interested in blocking the "evaluations" of request-response pairs. By setting this environment variable, the promp generations from api.promptfoo.app also stopped. What I mean is keeping the remote prompt generation but disabling the "llm-rubric" tasks to promptfoo endpoint (i.e. an environment variable like PROMPTFOO_DISABLE_PROMPTFOO_LLM_RUBRIC_ENDPOINT=true).
I am just trying to make suggestions, please ignore them if they are invalid. I love promptfoo overall. Thanks for creating such a great product.