https://promptfoo.dev/ logo
Display tool calls in evals
# ❓|questions
w
Hi there, I have a custom provider that runs a graph agent, however as part of the evals I would like to test what tools where called and with what arguments. How should I be saving this tool information? I'm using the javascript provider
t
@Will [Promptfoo]
w
You'll want to return information about your toolcalls under the
output
key of the object returned by your provider, and then use Javascript assertions to parse it https://www.promptfoo.dev/docs/configuration/expected-outputs/javascript/
w
Ahh that's fair. I suppose I can return JSON and do it that way, that'd work well! I just want to make sure I'm not stretching what promptfoo is supposed to be too much, as I know it's more about testing prompts, not entire applications
w
Promptfoo is like a swiss army knife feel free to stretch it however you like – if something breaks, let us know and we'll fix and often add first-class add support