https://promptfoo.dev/ logo
multiline variables in text files aren't valid JSO...
# ❓|questions
j
When you have variances of large system prompts for complex functionality that span multiple lines that you want to test and tweak against each other, it's much easier to do this in a separate text file, however this isn't valid JSON unless the line breaks are replaced with \n. Workaround is a using a javascript file for the variable and doing a replace.
Copy code
module.exports = function (varName, prompt, otherVars) {
  // Example logic to return a value based on the varName
  var output = `You are a helpful assistant with a bunch of rules
- you like bananas
- you hate rude people`
output = output.replace(/\n/g, "\\n");

  return { "output": output };
};
Might be worth updating promptfoo to replace new lines on every imported text file, unless there is a use case where you wouldn't want this.
i
Just to clarify, you're talking about when you have a
.json
file with a
{role, content}
array?
j
Thanks, sorry I wasn't more clear. I'm talking about variables from text files. The system just imports the text files raw, meaning I can't have multi-line text in the file, which is common in more advanced system prompts
Copy code
defaultTest:
    vars:
        system_message: file://../prompts/system_message.txt
also it would be SOOO nice if the those files would also be "watched" when running with "-w" flag since changes in complex system prompts require a ton of testing and tweaking.
appreciate your quick responses, it's a great platform and saving me a lot of time
also forgot to mention that I also have to double escape quotes.
Copy code
output = output.replace(/"/g, '\\"');
i
Got it, I see what you're saying. Let me see what I can do..
improves the watch functionality to watch var filepaths as well
more on this - currently, if the prompt file is JSON, then all vars (such as your
system_message
) will be properly escaped as strings. If this behavior isn't working for you for some reason, a small example would be really helpful
2 Views