https://promptfoo.dev/ logo
j
I'm curious if the MCP implementation may not match the documentation? I have promptfoo working for a few eval runs, but when trying to configure new tests using our remote MCP server I get errors. I'm following this: https://www.promptfoo.dev/docs/integrations/mcp/ using this in my config:
Copy code
providers:
  - id: openai:chat:gpt-4.1
    config:
      apiKey: sk-...
      mcp:
        enabled: true
        server:
          url: "https://[redacted].com/sse"
          name: "mcp-dev"
The server is working fine with other mcp clients (Cursor, Claude desktop), but when running tests they fail with:
Error: Failed to connect to MCP server mcp-dev: Remote MCP servers are not supported. Please use a local server file or npm package.
Copy code
$  npx -y promptfoo@latest --version
0.112.5
i
hmm let me ping the person who did the mcp implementation
u
Hey! That'll be added in the next release of promptfoo. Meanwhile, you can connect to your url by using
mcp-remote
(https://www.npmjs.com/package/mcp-remote), and configuring it like so:
Copy code
config:
      mcp:
        enabled: true
        server:
          command: npx
          args: ['mcp-remote', 'your-server.com/sse']
          name: mcp-dev
j
ah, that explains it! Also, FYI, passing headers to the MCP server will be important if not already implemented.
i
faizan added header support here btw 🙂 https://github.com/promptfoo/promptfoo/pull/4018