Skip to content
MCP server

Evals MCP Server

Author verifiable eval records through a draft→review→revise→submit loop with enforced…

Install
Pricing
Open source
Runs
Locally or Remotely
Version
0.2.0
Listing updated
5 days ago

About Evals MCP Server

Evals MCP Server is an open-source MCP server for ai models and llm tooling. It can run locally on your machine or as a hosted remote server.

Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.

Install

Run in a terminal

claude mcp add evals-mcp-server -- npx -y @cyanheads/evals-mcp-server

Add to claude_desktop_config.json

{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server"
      ]
    }
  }
}
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server"
      ]
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "evals-mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server"
      ]
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/evals-mcp-server"
      ]
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Configuration

  • EVALS_DATA_DIRRoot folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.
  • EVALS_REQUIRE_CONFIRMATIONWhen 'true', evals_submit_draft requests human confirmation before finalizing.
  • EVALS_DEFAULT_LICENSEDefault metadata.license applied when a draft omits one.
  • EVALS_CAPTURE_DIRDirectory of tool-call capture files; referenced EvalsIDs resolve to full dumps at submit.
  • MCP_LOG_LEVELSets the minimum log level for output (e.g., 'debug', 'info', 'warn').
  • MCP_TRANSPORT_TYPESelects the HTTP transport.
  • MCP_HTTP_HOSTThe hostname for the HTTP server.
  • MCP_HTTP_PORTThe port to run the HTTP server on.
  • MCP_HTTP_ENDPOINT_PATHThe endpoint path for the MCP server.
  • MCP_AUTH_MODEAuthentication mode to use: 'none', 'jwt', or 'oauth'.
  • EVALS_DATA_DIRRoot folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.
  • EVALS_REQUIRE_CONFIRMATIONWhen 'true', evals_submit_draft requests human confirmation before finalizing.
  • EVALS_DEFAULT_LICENSEDefault metadata.license applied when a draft omits one.
  • EVALS_CAPTURE_DIRDirectory of tool-call capture files; referenced EvalsIDs resolve to full dumps at submit.
  • MCP_LOG_LEVELSets the minimum log level for output (e.g., 'debug', 'info', 'warn').

Pricing

Open source

Evals MCP Server FAQ

Is Evals MCP Server free?

Yes. Evals MCP Server is open source, so you can use and self-host it for free.

How do I add Evals MCP Server to Claude Code?

Run this command in your terminal: claude mcp add evals-mcp-server -- npx -y @cyanheads/evals-mcp-server

What are the best alternatives to Evals MCP Server?

The closest alternatives to Evals MCP Server are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.

Where this comes from

  • Description: from the official MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers

Alternatives to Evals MCP Server

  1. MCP server for searching the Claude Code Ultimate Guide from any MCP client

    Runs locally
    • Open source
    • 6.5K GitHub stars
    • AI models and LLM tooling
  2. signafAIde

    MCP server

    Local MCP server to set up, verify and check coverage of signed agent action records

    Runs locally
    • 5 tools
    • Freemium
    • Security
  3. Toolkien

    MCP server

    One MCP gateway for all your MCP servers, with code mode, permissions and audit

    Not checked yet
    • 3 tools
    • Remote
    • Open source
    • AI models and LLM tooling
  4. Lugh

    MCP server

    Write, check and dry-run Lugh agent programs with policy, budgets and approvals

    Runs locally
    • 5 tools
    • Open source
    • AI models and LLM tooling
  5. HypePaper

    MCP server

    Research papers, benchmarks, citations and topic experts for AI assistants

    • 58 tools
    • Remote
    • Freemium
    • AI models and LLM tooling