I want to create a simple two-step chain in promptfoo - make calls to a prompt based on variables and then make calls to a second prompt based on results from the first prompt. I want to view resullts from both steps in the promptfoo output. It doesn't look like there's a way to refer to output variables from one prompt in another prompt. Is that correct? Looking at the 'Testing LLM chains' page, case #2 doesn't work because I do care about the intermediate result. Case #1 seems to be my only choice but does than mean I need to marshal the data out to a file in such a way that the second prompt can parse it back in?
i
ian
05/22/2024, 5:39 AM
Would the
_conversation
variable work for you? I know it's documented under OpenAI, but it actually works for any provider.
n
Nighthawk
05/22/2024, 6:11 AM
I hadn't noticed that but it wouldn't work for my case - I need to do a small transformation on the result from query 1 in order to get it in the same prompt as in my application. Maybe since I only have two steps, the stolution is to run it twice - once to the end of the first step to get the intermediate result, and once to the end of the second step to get the final result (as in case #2).
i
ian
05/22/2024, 8:38 AM
That might be a good stopgap. Perhaps the ideal solution would be a way to write the output of a previous test into a variable, with an optional transform
n
Nighthawk
05/22/2024, 3:31 PM
Can you do that with the current system?
i
ian
05/22/2024, 7:31 PM
Nope, but it can be added pretty easily.
ian
05/22/2024, 7:32 PM
I can share a concept PR and you can lmk if it'd solve your use case