quintens
09/05/2025, 6:12 PMprompts:
- "{{problem_description}}"
providers: # A single enabled custom Python provider
- id: file://promptfoo/promptfoo_classifier_provider.py
label: gpt5-only
config:
enabled_providers: ["gpt-5"]
decision_mode: "first"
taxonomy_name: "default"
include_debug_details: true
cache_enabled: true
tests: promptfoo/OSB_sample_data_mini.csv
# tests: promptfoo/OSB_sample_data.csv
defaultTest:
assert:
- type: javascript
value: |
// actual JS test here
this worked in 0.118.0, seems to fail in 0.118.3. downgrading to 0.118.0 seems to get things working again, so maybe just a bug? i didn't see a related issue in GitHub yet and it's also possible I just have weird syntax that I should fix[Promptfoo] Steve
09/05/2025, 8:51 PMquintens
09/05/2025, 8:55 PMconfig parameter which on the 0.118.0 version is passing through fine to my custom provider.py[Promptfoo] Steve
09/05/2025, 8:57 PM[Promptfoo] Steve
09/05/2025, 8:57 PMquintens
09/05/2025, 9:04 PM[Promptfoo] Steve
09/05/2025, 10:15 PMquintens
09/06/2025, 4:36 PM