https://promptfoo.dev/ logo
Recommended way to enter unique scoring for each v...
# ❓|questions
m
Hey, I have a question for which I'm looking for best practices. If answers are in the docs, I didn't find them. How to input my data where I have questions and expected answers, which are different for each question? Ideally I'd load them from files / folder structure, without entering each path to the yaml. The CSV input seems promising. I can also generate the yaml programmatically. But maybe there is built-in way? I am testing a RAG pipeline. In one file I have a prompt that I ask from a custom provider that implements RAG. I have instructions for scoring the response in another file, which I run through a scoring model. How to best specify this when the input and output pairs are always different?
Actually to clarify terminology, I need a way to associate unique scoring per variable, the prompt stays constant.
i
hmm, unfortunately what I've seen people do is generate their own yaml/json. I'm open to suggestions here, as you're not the first person to bring up something like this
some people also use scenarios for setups like this