Skip to content
MCP server

The Aggregate — LLM benchmark aggregate

Cross-benchmark LLM rankings with significance checks, free and read-only

Install
Pricing
Free
Platforms
API
Runs
Remotely
Version
1.0.2
Listing updated
1 day ago

About The Aggregate — LLM benchmark aggregate

The Aggregate — LLM benchmark aggregate is a free MCP server for ai models and llm tooling. It runs as a hosted remote server, so there is nothing to install locally.

Rather than reporting one leaderboard, The Aggregate fuses many public LLM benchmark leaderboards into a single ranking using an item response theory model. Each model carries an Aggregate Rating with a standard error, rebuilt daily, so a head-to-head comparison can say whether a rating gap is statistically meaningful.

Eight tools cover the leaderboard, model and benchmark search, per-model and per-benchmark detail, comparison, a prediction-duel standing that pits the site's model against frontier LLMs at forecasting new scores, and a methodology note with citation guidance. Per-benchmark scores link back to the upstream leaderboards.

The endpoint is stateless Streamable HTTP, needs no key, and is rate-limited per IP. Published numbers should credit The Aggregate and the upstream leaderboards.

Install

Run in a terminal

claude mcp add --transport http the-aggregate https://theaggregate.ai/mcp

Add to Settings → Connectors → Add custom connector

https://theaggregate.ai/mcp
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "the-aggregate": {
      "url": "https://theaggregate.ai/mcp"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "the-aggregate": {
      "type": "http",
      "url": "https://theaggregate.ai/mcp"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "the-aggregate": {
      "serverUrl": "https://theaggregate.ai/mcp"
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "the-aggregate": {
      "type": "http",
      "url": "https://theaggregate.ai/mcp"
    }
  }
}

Tools this server provides (8)

Taken from the project documentation; the running server may differ.

get_leaderboard
Paged fused cross-benchmark ranking
search_models
Find ranked models by name or provider
get_model
One model: rank, rating, description, cost per task
compare_models
Head-to-head with significance check
search_benchmarks
Find benchmarks in the aggregate
get_benchmark
One benchmark with source link and stats
get_prediction_duel
Prediction standings versus frontier LLMs
about_the_aggregate
Method, coverage and how to cite

What it does

Fused leaderboard
One ranking built from public benchmarks via an IRT model.
Rating with error
Aggregate Ratings come with standard errors and daily rebuilds.
Significance check
compare_models tests whether the rating gap is meaningful.
Benchmark detail
get_benchmark returns source link, IRT stats, skills and top models.
Prediction duel
Shows how the model fares against frontier LLMs at predicting new scores.
Source links
Responses link the upstream leaderboards.

Good for

  • Pick a model by cross-benchmark standing
  • Compare two models head to head
  • Look up what a benchmark measures
  • Cite model rankings with sources

Pricing

Free

Free, read-only

Prices change. Check theaggregate.ai before you buy.

The Aggregate — LLM benchmark aggregate FAQ

Do I need a key?

No key or sign-up is needed.

How often does data update?

Ratings are rebuilt daily.

Is it rate-limited?

Yes, per IP.

Do I have to credit anyone?

Credit The Aggregate and the upstream leaderboards when publishing numbers.

Is The Aggregate — LLM benchmark aggregate free?

Yes. The Aggregate — LLM benchmark aggregate is free to use.

How do I add The Aggregate — LLM benchmark aggregate to Claude Code?

Run this command in your terminal: claude mcp add --transport http the-aggregate https://theaggregate.ai/mcp

What are the best alternatives to The Aggregate — LLM benchmark aggregate?

The closest alternatives to The Aggregate — LLM benchmark aggregate are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.

Alternatives to The Aggregate — LLM benchmark aggregate

  1. MCP server for searching the Claude Code Ultimate Guide from any MCP client

    Runs locally
    • Open source
    • 6.5K GitHub stars
    • AI models and LLM tooling
  2. signafAIde

    MCP server

    Local MCP server to set up, verify and check coverage of signed agent action records

    Runs locally
    • 5 tools
    • Freemium
    • Security
  3. Toolkien

    MCP server

    One MCP gateway for all your MCP servers, with code mode, permissions and audit

    Not checked yet
    • 3 tools
    • Remote
    • Open source
    • AI models and LLM tooling
  4. Lugh

    MCP server

    Write, check and dry-run Lugh agent programs with policy, budgets and approvals

    Runs locally
    • 5 tools
    • Open source
    • AI models and LLM tooling
  5. HypePaper

    MCP server

    Research papers, benchmarks, citations and topic experts for AI assistants

    • 58 tools
    • Remote
    • Freemium
    • AI models and LLM tooling