Skip to content
MCP server

Agent Referee

Neutral tie-breaks between AI agents and independent review of an agent's output

Runs locallyStatus and history
Install
Pricing
Freemium
Platforms
CLI, API
Runs
Locally
Version
0.4.0
Listing updated
2 days ago

About Agent Referee

Agent Referee is a freemium MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

Two agents from different owners sometimes reach a deadlock that neither should settle alone, and a single agent's claim that its work is correct is weak evidence. Agent Referee covers both, returning an Ed25519-signed receipt that anyone can check offline against a pinned key.

For disputes there are tiers from a weighted coin flip and picking one of 2 to 20 options, through rock-paper-scissors, to AI judges applying rules both sides agreed to. Judge tiers use one, two or three judges from different AI companies, and the receipt names every model and version. For solo work, sealed review sends an agent's output and task to independent reviewers who do not see each other's verdicts and return pass, needs changes, fail or cannot assess, with the problems found. The receipt holds a fingerprint of the output, not the output.

The free tools are a demo match, receipt verification and a daily allowance of free reviews, which the README gives as 3 single-reviewer reviews and 1 three-reviewer review per address per day. Paid calls use USDC on Base over x402 from a wallet you configure, with a default spend limit of $1.00 per UTC day. The README says results are advisory and not for legal or contractual disputes.

Install

Run in a terminal

claude mcp add agent-referee --env AGENT_REFEREE_PRIVATE_KEY='<AGENT_REFEREE_PRIVATE_KEY>' -- npx -y agent-referee-mcp

Add to claude_desktop_config.json

{
  "mcpServers": {
    "agent-referee": {
      "command": "npx",
      "args": [
        "-y",
        "agent-referee-mcp"
      ],
      "env": {
        "AGENT_REFEREE_PRIVATE_KEY": "<AGENT_REFEREE_PRIVATE_KEY>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "agent-referee": {
      "command": "npx",
      "args": [
        "-y",
        "agent-referee-mcp"
      ],
      "env": {
        "AGENT_REFEREE_PRIVATE_KEY": "<AGENT_REFEREE_PRIVATE_KEY>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "agent-referee": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "agent-referee-mcp"
      ],
      "env": {
        "AGENT_REFEREE_PRIVATE_KEY": "<AGENT_REFEREE_PRIVATE_KEY>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "agent-referee": {
      "command": "npx",
      "args": [
        "-y",
        "agent-referee-mcp"
      ],
      "env": {
        "AGENT_REFEREE_PRIVATE_KEY": "<AGENT_REFEREE_PRIVATE_KEY>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "agent-referee": {
      "command": "npx",
      "args": [
        "-y",
        "agent-referee-mcp"
      ],
      "env": {
        "AGENT_REFEREE_PRIVATE_KEY": "<AGENT_REFEREE_PRIVATE_KEY>"
      }
    }
  }
}

Configuration

  • AGENT_REFEREE_PRIVATE_KEYOptional. Private key of a dedicated Base wallet holding a few dollars of USDC, for paid sealed reviews (also start the server with the x402-fetch package: npx -y -p agent-referee-mcp -p x402-fetch agent-referee-mcp). Free tools work without it.
  • AGENT_REFEREE_DAILY_LIMIT_USDOptional. Most this server may spend per UTC day (default 1.00; 0 turns paid calls off).

Tools this server provides (6)

Taken from the project documentation; the running server may differ.

demo_match
Run a complete demo tie-break in one call
tiebreak
Settle a deadlock with another agent
sealed_review
Independent reviewers check an agent's output and return a signed receipt
verify_receipt
Check a receipt offline against the pinned signing key
review_result
Return a sealed review that was still running
saved_receipts
List receipts saved locally and today's spend

What it does

Signed receipts
Decisions and reviews come with Ed25519-signed receipts checkable offline.
Multi-company judges
Judge and review tiers can draw models from different AI companies and name them in the receipt.
Sealed review
Independent reviewers check one agent's output without seeing each other's verdicts.
Free demo and reviews
A demo match and a daily allowance of free reviews need no wallet.
Spend cap
A daily limit, default $1.00, caps what the server may spend; 0 turns paid calls off.
Receipt verification
Confirms a receipt matches exactly the task and output it claims.

Good for

  • Settle a deadlock between two agents owned by different parties
  • Get an independent check of an agent's output before handing it on
  • Verify that a receipt belongs to a specific piece of work
  • Try a tie-break with the free demo

Pricing

Freemium

Free demo and daily free reviews; paid tiers per call in USDC, from $0.05 per review

Prices change. Check agentreferee.ai before you buy.

Agent Referee FAQ

Do I need a wallet?

Not for the free tools. Paid tiers need a Base wallet holding USDC.

Is it suitable for legal disputes?

No. The README says it is for technical evaluation and results are advisory.

What does a review cost?

From $0.05 for one reviewer and $0.20 for three, per the README, after the free daily allowance.

Does the receipt contain my output?

No, it holds a fingerprint of the output.

Is Agent Referee free?

Agent Referee has a free plan, with paid plans for more usage or features.

How do I add Agent Referee to Claude Code?

Run this command in your terminal: claude mcp add agent-referee --env AGENT_REFEREE_PRIVATE_KEY='<AGENT_REFEREE_PRIVATE_KEY>' -- npx -y agent-referee-mcp

What are the best alternatives to Agent Referee?

The closest alternatives to Agent Referee are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.

Alternatives to Agent Referee

  1. MCP server for searching the Claude Code Ultimate Guide from any MCP client

    Runs locally
    • Open source
    • 6.5K GitHub stars
    • AI models and LLM tooling
  2. signafAIde

    MCP server

    Local MCP server to set up, verify and check coverage of signed agent action records

    Runs locally
    • 5 tools
    • Freemium
    • Security
  3. Toolkien

    MCP server

    One MCP gateway for all your MCP servers, with code mode, permissions and audit

    Not checked yet
    • 3 tools
    • Remote
    • Open source
    • AI models and LLM tooling
  4. Lugh

    MCP server

    Write, check and dry-run Lugh agent programs with policy, budgets and approvals

    Runs locally
    • 5 tools
    • Open source
    • AI models and LLM tooling
  5. HypePaper

    MCP server

    Research papers, benchmarks, citations and topic experts for AI assistants

    • 58 tools
    • Remote
    • Freemium
    • AI models and LLM tooling