Evals MCP Server
Author verifiable eval records through a draft→review→revise→submit loop with enforced…
- Pricing
- Open source
- Runs
- Locally or Remotely
- Version
- 0.2.0
- Listing updated
- 5 days ago
About Evals MCP Server
Evals MCP Server is an open-source MCP server for ai models and llm tooling. It can run locally on your machine or as a hosted remote server.
Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.
Install
Run in a terminal
claude mcp add evals-mcp-server -- npx -y @cyanheads/evals-mcp-serverAdd to claude_desktop_config.json
{
"mcpServers": {
"evals-mcp-server": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"evals-mcp-server": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to .vscode/mcp.json
{
"servers": {
"evals-mcp-server": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"evals-mcp-server": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"evals-mcp-server": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Using several servers? Build one config file with the MCP config generator.
Configuration
EVALS_DATA_DIRRoot folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.EVALS_REQUIRE_CONFIRMATIONWhen 'true', evals_submit_draft requests human confirmation before finalizing.EVALS_DEFAULT_LICENSEDefault metadata.license applied when a draft omits one.EVALS_CAPTURE_DIRDirectory of tool-call capture files; referenced EvalsIDs resolve to full dumps at submit.MCP_LOG_LEVELSets the minimum log level for output (e.g., 'debug', 'info', 'warn').MCP_TRANSPORT_TYPESelects the HTTP transport.MCP_HTTP_HOSTThe hostname for the HTTP server.MCP_HTTP_PORTThe port to run the HTTP server on.MCP_HTTP_ENDPOINT_PATHThe endpoint path for the MCP server.MCP_AUTH_MODEAuthentication mode to use: 'none', 'jwt', or 'oauth'.EVALS_DATA_DIRRoot folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.EVALS_REQUIRE_CONFIRMATIONWhen 'true', evals_submit_draft requests human confirmation before finalizing.EVALS_DEFAULT_LICENSEDefault metadata.license applied when a draft omits one.EVALS_CAPTURE_DIRDirectory of tool-call capture files; referenced EvalsIDs resolve to full dumps at submit.MCP_LOG_LEVELSets the minimum log level for output (e.g., 'debug', 'info', 'warn').
Pricing
Open source
Evals MCP Server FAQ
Is Evals MCP Server free?
Yes. Evals MCP Server is open source, so you can use and self-host it for free.
How do I add Evals MCP Server to Claude Code?
Run this command in your terminal: claude mcp add evals-mcp-server -- npx -y @cyanheads/evals-mcp-server
What are the best alternatives to Evals MCP Server?
The closest alternatives to Evals MCP Server are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.
Where this comes from
- Description: from the official MCP Registry entry.
- Install steps: from the official MCP Registry entry.
Alternatives to Evals MCP Server

Claude Code Ultimate Guide
MCP serverMCP server for searching the Claude Code Ultimate Guide from any MCP client
Runs locally- Open source
- 6.5K GitHub stars
- AI models and LLM tooling

signafAIde
MCP serverLocal MCP server to set up, verify and check coverage of signed agent action records
Runs locally- 5 tools
- Freemium
- Security

Toolkien
MCP serverOne MCP gateway for all your MCP servers, with code mode, permissions and audit
Not checked yet- 3 tools
- Remote
- Open source
- AI models and LLM tooling
Lugh
MCP serverWrite, check and dry-run Lugh agent programs with policy, budgets and approvals
Runs locally- 5 tools
- Open source
- AI models and LLM tooling
HypePaper
MCP serverResearch papers, benchmarks, citations and topic experts for AI assistants
- 58 tools
- Remote
- Freemium
- AI models and LLM tooling

One compact entry point that finds, describes and runs hundreds of pay-per-call data tools
- 3 tools
- Remote
- Freemium
- Developer tools