plyra
06/23/2024, 4:56 PMian
06/23/2024, 5:34 PMplyra
06/23/2024, 5:47 PM{"what's 2 + 2?", "math", [2, 4, 6, 8, 10], B}plyra
06/23/2024, 5:49 PMB with 4 anyway, whatever works easier.ian
06/23/2024, 5:57 PM{{variableName}} you use in the prompt, and then use the __expected column to set the expected correct answer output.
I assume your prompt will do something like ask it to output the multiple choice answer, so you can use starts-with:B in the __expected column for example.
Also happy to walk you through the yaml config, which is more flexible than CSV (may be necessary if you're doing something more complicated)