joemiller88
03/25/2024, 9:17 PMproviders:
- customProvider.js:
id: custom-provider-hightemp
config:
temperature: 1.0
- customProvider.js:
id: custom-provider-lowtemp
config:
temperature: 0
I can confirm that this does not work as it actually uses the id to look for the file so the `id`has to match the filename less the extension
With the above code, you will get
Error: Cannot find module '/Users/me/dev/promptfoo-evals/tests/example-tool-call/custom-provider-hightem'
The other issue with this is that it doesn't allow the file to be anywhere other than the current directory
What does work is referencing it from the command line with a path, but then you're limited and can't pass optionsian
03/26/2024, 2:16 PMian
03/26/2024, 5:49 PMian
03/26/2024, 5:49 PM