https://promptfoo.dev/ logo
Python provider id
# ❓|questions
w
How can I distinguish multiple instances of custom python provider by id (e.g. with different params)? I'm unable to set provider key to
python:provider.py
and then set id separately
i
Can you try something like this:
Copy code
yaml
providers:
  - python:provider.py:
      id: custom-provider-1
      config:
        temperature: 0
  - python:provider.py
      id: custom-provider-2
      config:
        temperature: 1
w
I'm getting following error:
Copy code
node:internal/modules/cjs/loader:1077
  const err = new Error(message);
              ^

Error: Cannot find module '/home/sagemaker-user/promptfoo/custom-provider-1'
Require stack:
- /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/providers.js
- /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/matchers.js
- /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/assertions.js
- /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/main.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:121:18)
    at /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/providers.js:295:132
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async loadApiProvider (/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/providers.js:295:32)
    at async Promise.all (index 0) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/providers.js',
    '/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/matchers.js',
    '/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/assertions.js',
    '/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/main.js'
  ]
}
But it looks like this works:
Copy code
yaml
providers:
  - custom-provider-1:
      id: python:provider.py
      config:
        temperature: 0
  - custom-provider-2:
      id: python:provider.py
      config:
        temperature: 1
But I can't see "custom-provider-..." in outputs
To be more specific, in generated HTML