https://promptfoo.dev/ logo
How to add assert statemenst in case of google she...
# ❓|questions
a
The docs show examples of using Google Sheets; however, they don't mention how to use assert and llm rubrics.
Copy code
yaml
prompts: [prompt1.txt, prompt2.txt]
providers: [openai:gpt-3.5-turbo, vertex:gemini-pro]
tests: https://docs.google.com/spreadsheets/d/1eqFnv1vzkPvS7zG-mYsqNDwOzvSaiIAsKB3zKg9H18c/edit?usp=sharing
If we're add the tests object with link to sheets, where do I specify the llm rubric or other assert statements?
i
Have a look at this: https://promptfoo.dev/docs/configuration/expected-outputs/#load-assertions-from-csv TLDR prefix the column with
__expected
to include assertions. For example:
Copy code
__expected1,__expected2
llm-rubric: does not apologize,contains:foobar
a
I'll check it out. Thanks Also, how do you change the API key variable name when using a different providers such as Azure for the LLM rubric? We have a diff deployment for GPT4 and 16k. I thought I could change the env variable with
apiKeyEnvvar
but it's not working. The same azure variable won't work with both.
Copy code
yaml
defaultTest:
  options:
    provider:
      id: azureopenai:chat:evaluation-us-south-central
      config:
        apiHost: evaluation-south-central-us.openai.azure.com
        temperature: 0.2
        apiVersion: 2024-02-15-preview
        apiKey: 'HARDCODING KEY HERE WORKS>'
        apiKeyEnvar: AZURE_EVALUATION_API_KEY <❌ THIS DOESNT>
i
apiKeyEnvar currently not supported for Azure OpenAI, this PR adds it: https://github.com/promptfoo/promptfoo/pull/628