Skip to content
MCP server

Agent Eval

Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior…

Runs locallyStatus and history
Install
Pricing
Open source
Runs
Locally
Version
0.1.8
Listing updated
6 days ago

About Agent Eval

Agent Eval is an open-source MCP server. It runs locally on your machine and is started by your AI client.

Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.

Install

Run in a terminal

claude mcp add agent-eval -- uvx agent-regress-cli

Add to claude_desktop_config.json

{
  "mcpServers": {
    "agent-eval": {
      "command": "uvx",
      "args": [
        "agent-regress-cli"
      ]
    }
  }
}
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "agent-eval": {
      "command": "uvx",
      "args": [
        "agent-regress-cli"
      ]
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "agent-eval": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "agent-regress-cli"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "agent-eval": {
      "command": "uvx",
      "args": [
        "agent-regress-cli"
      ]
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "agent-eval": {
      "command": "uvx",
      "args": [
        "agent-regress-cli"
      ]
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Pricing

Open source

Agent Eval FAQ

Is Agent Eval free?

Yes. Agent Eval is open source, so you can use and self-host it for free.

How do I add Agent Eval to Claude Code?

Run this command in your terminal: claude mcp add agent-eval -- uvx agent-regress-cli

Where this comes from

  • Description: from the official MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers