asterwork
11/15/2023, 4:14 PMyaml
tests:
- description: "does not describe self as an AI, model, or chatbot"
assert:
- type: llm-rubric
value: does not describe self as an AI, model, or chatbot
# need to figure out how to specify the azure provider here
provider: azureopenai:chat:gptDeployment
error
cmd
[FAIL] Error: Azure OpenAI API host must be set
Error: Azure OpenAI API host must be set
at AzureOpenAiChatCompletionProvider.callApi (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\providers\azureopenai.js:174:19)
at matchesLlmRubric (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\matchers.js:164:38)
at async runAssertion (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\assertions.js:507:17)
at async runAssertions (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\assertions.js:58:24)
at async Evaluator.runEval (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\evaluator.js:208:37)
at async C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\evaluator.js:465:25
then
yaml
tests:
- description: "does not describe self as an AI, model, or chatbot"
assert:
- type: llm-rubric
value: does not describe self as an AI, model, or chatbot
# need to figure out how to specify the azure provider here
provider:
- id: azureopenai:chat:gptDeployment
config:
apiHost: "myendpoint.openai.azure.com"
gets error [next message]asterwork
11/15/2023, 4:14 PMcmd
[FAIL] Error: Invariant failed: Provider supplied to assertion must have an id
Error: Invariant failed: Provider supplied to assertion must have an id
at invariant (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\node_modules\tiny-invariant\dist\tiny-invariant.cjs.js:14:11)
at loadFromProviderOptions (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\matchers.js:39:34)
at getGradingProvider (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\matchers.js:59:31)
at getAndCheckProvider (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\matchers.js:68:31)
at matchesLlmRubric (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\matchers.js:163:31)
at runAssertion (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\assertions.js:507:55)
at runAssertions (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\assertions.js:58:30)
at Evaluator.runEval (C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\evaluator.js:208:74)
at async C:\Users\York Laptop 001\AppData\Roaming\npm\node_modules\promptfoo\dist\src\evaluator.js:465:25
the id is right there-- how do i set this properly?asterwork
11/15/2023, 4:18 PMdefaultTest.optionsasterwork
11/15/2023, 4:19 PMyaml
defaultTest:
options:
provider:
id: azureopenai:chat:gptDeployment
config:
apiHost: "myendpoint.openai.azure.com"
tests:
- description: "does not describe self as an AI, model, or chatbot"
assert:
- type: llm-rubric
value: does not describe self as an AI, model, or chatbotian
11/15/2023, 7:30 PMprovider an array in the original example, instead of an object.
Before:
yaml
tests:
- description: "does not describe self as an AI, model, or chatbot"
assert:
- type: llm-rubric
value: does not describe self as an AI, model, or chatbot
# need to figure out how to specify the azure provider here
provider:
- id: azureopenai:chat:gptDeployment
config:
apiHost: "myendpoint.openai.azure.com"
Probable fix (remove - before provider id):
yaml
tests:
- description: "does not describe self as an AI, model, or chatbot"
assert:
- type: llm-rubric
value: does not describe self as an AI, model, or chatbot
# need to figure out how to specify the azure provider here
provider:
id: azureopenai:chat:gptDeployment
config:
apiHost: "myendpoint.openai.azure.com"asterwork
11/15/2023, 7:31 PMian
11/15/2023, 7:31 PMasterwork
11/15/2023, 7:32 PMian
11/15/2023, 7:32 PMasterwork
11/15/2023, 7:33 PMasterwork
11/15/2023, 7:36 PM