https://promptfoo.dev/ logo
Custom Providers only works if file is in the curr...
# ❓|questions
j
Trying to use custom providers and the example code shows this
Copy code
providers:
   - customProvider.js:
       id: custom-provider-hightemp
       config:
         temperature: 1.0
   - customProvider.js:
       id: custom-provider-lowtemp
       config:
         temperature: 0
I can confirm that this does not work as it actually uses the
id
to look for the file so the `id`has to match the filename less the extension With the above code, you will get
Copy code
Error: Cannot find module '/Users/me/dev/promptfoo-evals/tests/example-tool-call/custom-provider-hightem'
The other issue with this is that it doesn't allow the file to be anywhere other than the current directory What does work is referencing it from the command line with a path, but then you're limited and can't pass options
i
You're right, looks like provider ID should be resolved relative to config rather than current working directory
will be in the next published release
3 Views