Will
04/26/2024, 11:13 PMprompts:
- prompts/few-shot-json.js
- prompts/true-false-json.js
providers:
- id: openai:chat:llama3-70b-8192
label: 'GROQ-LLAMA-70B'
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
response_format: { type: 'json_object' }
- id: openai:chat:llama3-8b-8192
label: 'GROQ-LLAMA-8B'
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
response_format: { type: 'json_object' }
I tried following the docs instruction: To reference a specific function in your prompt file, use the following syntax: filename.js:functionName: , but this didn't work. The code runs without error but 0 evals are run.
prompts:
- prompts/few-shot-json.js:few-shot-prompt
- prompts/true-false-json.js:true-false-prompt
providers:
- id: openai:chat:llama3-70b-8192
label: 'GROQ-LLAMA-70B'
prompts:
- prompts/few-shot-json.js:few-shot-prompt
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
- id: openai:chat:llama3-8b-8192
label: 'GROQ-LLAMA-8B'
prompts:
- prompts/true-false-json.js:true-false-prompt
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
When I remove the prompts: at the top, it errors out and says
throw new Error(`There are no prompts in ${JSON.stringify(promptPathOrGlobs)}`);
^
Error: There are no prompts in {}
at readPrompts (/Users/wl/.nvm/versions/node/v18.17.0/lib/node_modules/promptfoo/dist/src/prompts.js:273:15)
Can anyone provide an example? That would be super helpful. Thanks!ian
04/27/2024, 2:01 AMprompts section, and then reference that name in providers.promptsWill
04/27/2024, 2:54 PM/Users/wl/.nvm/versions/node/v18.17.0/lib/node_modules/promptfoo/dist/src/util.js:247
if (relativePath.startsWith('file://')) {
^
TypeError: relativePath.startsWith is not a function
This code below throws the error above. Do you see something wrong with the way I'm doing it here?
prompts:
- prompts/true-false.js: true-false-prompt
- prompts/many-shot.js: many-shot-prompt
providers:
- id: openai:chat:llama3-70b-8192
prompts:
- true-false-prompt
label: 'GROQ-LLAMA-70B'
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
response_format: { type: 'json_object' }
- id: openai:chat:llama3-8b-8192
prompts:
- many-shot-prompt
label: 'GROQ-LLAMA-8B'
config:
temperature: 0
apiBaseUrl: https://api.groq.com/openai/v1
apiKeyEnvar: GROQ_API_KEY
response_format: { type: 'json_object' }ian
04/28/2024, 5:48 PMprompts should not be an array with this format. You should remove the `- `:
yaml
prompts:
prompts/true-false.js: true-false-prompt
prompts/many-shot.js: many-shot-prompt
This bit me the other day actually, and the next release includes a better error messageWill
04/28/2024, 10:20 PM