https://promptfoo.dev/ logo
Python provider issue
# ❓|questions
w
I wa testing reranking using github.com/bclavie/RAGatouille. Under the hood it writes some text to stdout/stdin, and I think it breaks my pipeline, here is the error message:
Copy code
SyntaxError: Unexpected token P in JSON at position 1 SyntaxError: Unexpected token P in JSON at position 1 at JSON.parse (<anonymous>) at PythonProvider.callApi (/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/providers/pythonCompletion.js:41:33) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Evaluator.runEval (/home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/evaluator.js:157:30) at async /home/sagemaker-user/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/evaluator.js:477:25
I'm unable to debug (tried to redirect all stdour & stderr to devnull, and when running the python script itself I'm not getting any text in bash, but when setting --verbose in promptfoo, I can't get relevant data for test case that outputs error above)
i
hmm, I think I can guess the issue. The provider expects the last line outputted by the script execution wrapper to be the python caller's JSON output: This would suggest that your script is writing to stdout after
call_api
has concluded. Is it possible it spawned some async task?