Hi
@[Promptfoo] Faizan , I'm trying to evaluate agent workflow.. I'm using provider as below:
providers: [
{
id: 'openai:chat:gpt-4.1',
config: {
mcp: {
enabled: true,
server: {
url: 'http://localhost:8091/sse',
name: 'local-server',
},
},
},
},
]
and for the given prompt this provider only returns me the tool information.. it doesnot execute the tool.. My given prompt depends on more than one tool execution example..."Check if Contact page exists if not present create a fresh page".. for this prompt
check_page_exists tool and
create_page should be called based on the
check_page_exists tool response.
I want a provider similar to openai:responses model... based on the prompt it will fetch the tools and execute and return the response to the host...
I'm looking some thing like this
https://github.com/promptfoo/promptfoo/blob/main/examples/openai-mcp/promptfooconfig.approval.yaml where responses model is getting tool info and executing the tool from the global mcp server and returning the response back to the client.. but the mcp server should be available publicly over the internet this is the limitation for responses model.. I want the same for the remote mcp server and that too multi-turn kind of conversation..