https://promptfoo.dev/ logo
Invariant failed: Invalid prompt (0.89.3 <= versio...
# ❓|questions
s
Hello, I ran test file below.
Copy code
providers:
  - id: vertex:gemini-1.5-pro-001
    prompts:
      - vertexai_prompts

prompts:
  - id: file://prompts/vertexai_prompts.json
    label: vertexai_prompts

defaultTest:
  vars:
    vertexai_instructions: file://instructions/get_vertexai_instructions.py

tests:
  - file://tests/general_tests.yaml
And I get this error over 0.89.3 version.
Copy code
/home/python/.npm/_npx/81bbc6515d992ace/node_modules/tiny-invariant/dist/tiny-invariant.cjs.js:14
    throw new Error(value);
          ^

Error: Invariant failed: Invalid prompt
    at invariant (/home/python/.npm/_npx/81bbc6515d992ace/node_modules/tiny-invariant/dist/tiny-invariant.cjs.js:14:11)
    at /home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/config.js:320:46
    at Array.forEach (<anonymous>)
    at /home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/config.js:319:28
    at Array.forEach (<anonymous>)
    at readConfigs (/home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/config.js:311:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async resolveConfigs (/home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/config.js:379:22)
    at async runEvaluation (/home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/commands/eval.js:73:55)
    at async doEval (/home/python/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/commands/eval.js:268:5)
and It works well on 0.89.2 version. Is there a change with the way of using 'prompts'?
v
@stkim03_96161 Can you try adding a raw: '' key to your prompt object as a workaround?
Copy code
prompts:
  - id: file://prompts/vertexai_prompts.json
    label: vertexai_prompts
    raw: ''
We have this issue logged in GitHub if you want to follow along. We are working on the release in the next day. https://github.com/promptfoo/promptfoo/issues/1738
s
@vanessa.promptfoo_45142 yeah right. Your trick works for me too! But the problem is that it doesn't seem to create prompts properly (
file://prompts/vertexai_prompts.json
this part). and it works on 0.89.2. do you have same situation?
j
@User picking up on this thread, when using prompt functions, it executes all variations. So in the example below, I get 4 outputs instead of the two I was hoping for. Any workarounds?
Copy code
prompts:
  - id: file://../../../prompts/proactive-eval/prompt.js
    label: "proactive-eval-prompt"
  - id: file://../../../prompts/proactive-message/prompt.js
    label: "proactive-message-prompt"

providers:
  - id: file://../../../prompts/proactive-eval/provider.yaml
    prompts: 
      - "proactive-eval-prompt"
  - id: file://../../../prompts/proactive-message/provider.yaml
    prompts: 
      - "proactive-message-prompt"