https://promptfoo.dev/ logo
xml output error
# ❓|questions
c
When I do
promptfoo eval -c config.yaml --output result.xml
, I got the following error message. Is this a known issue?
Copy code
/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:268
      textValue = textValue.replace(entity.regex, entity.val);
                            ^
TypeError: textValue.replace is not a function
    at Builder.replaceEntitiesValue (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:268:29)
    at Builder.buildTextValNode (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:252:22)
    at Builder.j2x (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:116:23)
    at Builder.processTextOrObjNode (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:181:23)
    at Builder.j2x (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:140:32)
    at Builder.processTextOrObjNode (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:181:23)
    at Builder.j2x (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:165:21)
    at Builder.processTextOrObjNode (/opt/homebrew/Cellar/promptfoo/0.117.4/libexec/lib/node_modules/promptfoo/node_modules/fast-xml-parser/src/xmlbuilder/json2xml.js:181:23)
    at Builder.j2x
... truncating the rest because message is too long
Node.js v24.4.1
t
@Will [Promptfoo] mind answering please?
w
Hi @cyh0530, can you share your
config.yaml
? Thanks
c
I think the problem is python prompts. Switching to inline or json prompt won't have issue. config.yaml
Copy code
description: "Response evaluation"

prompts:
  - file://sample.py:get_prompt

providers:
  - "openrouter:openai/gpt-4.1"

tests:
  - description: Response evaluation
    assert:
      - type: model-graded-closedqa
        value: The assistant answers the question.
sample.py
Copy code
def get_prompt(_):
    return [
        {
            "role": "user",
            "content": "What's the capital of France?",
        },
    ]
w
I'm not able to reproduce it. Can you update to the latest version of
promptfoo
(1.117.5) and then try re-running the command with the
--no-cache
and
--verbose
flags appended to it?
c
looks like the issue is resolved after this [fix](https://github.com/promptfoo/promptfoo/pull/5194) and released in 0.117.5 . Thank you