https://promptfoo.dev/ logo
Javascript eval scripts not working
# ❓|questions
c
Title: Error: Unsupported file type when running assertions on generated JavaScript file Describe the bug When running assertions in Promptfoo, I encountered an error related to an unsupported file type. The error message specifically points to a JavaScript file (
assert_1.js
) generated in my test setup.
Copy code
Error: Unsupported file type: C:<path_to_my_custom_assert_file>\js\assert_1.js
    at processFileReference (<path_to_local_promptfoo>\promptfoo\src\assertions\utils.ts:67:11)
    at runAssertion (<path_to_local_promptfoo>\promptfoo\src\assertions\index.ts:183:45)
    at <path_to_local_promptfoo>promptfoo\src\assertions\index.ts:358:28
    at <path_to_local_promptfoo>\promptfoo\node_modules\async\dist\async.js:151:38
    at replenish (<path_to_local_promptfoo>\promptfoo\node_modules\async\dist\async.js:447:21)
    at <path_to_local_promptfoo>\promptfoo\node_modules\async\dist\async.js:452:13
    at Object.eachOfLimit (<path_to_local_promptfoo>promptfoo\node_modules\async\dist\async.js:478:36)
    at <path_to_local_promptfoo>\promptfoo\node_modules\async\dist\async.js:220:25
    at new Promise (<anonymous>)
    at Object.awaitable (<path_to_local_promptfoo>\promptfoo\node_modules\async\dist\async.js:215:20)
I created my custom assertions mechanism so i use
js
files like this:
Copy code
- assert:
  - type: javascript
    value: file:///<path_to_file>/js/assert_1.js
I recently updated my app from 0.95.0 to 0.104.0 and it stopped working. I use Node 20.18.3
m
@czekis. can you please share a sample config to reproduce your issue?
c
@michaelmichaelmichael hi! found a problem. I was using absolute paths, while it is recommended to use relative. NOTE: absolute paths work on linux, dont work on windows machines. Had to standarize to relative.
but in 0.95.0 absolute pathing for files worked just fine
m
@czekis. I think this PR partially resolves the abspath issue https://github.com/promptfoo/promptfoo/pull/3060 but it wasn't implemented everywhere. I'll add this to my backlog. Glad you figured out that relative paths work