Skip to content
MCP server

LLMBenchmark.ai

Free MCP server and REST API that recommends LLMs for a task using benchmark scores

Install

LLMBenchmark.ai is a free MCP server for ai models and llm tooling. It can run locally on your machine or as a hosted remote server.

Pricing
Free
Platforms
API, CLI
Runs
Locally or Remotely
Version
1.0.1
Listing updated

Similar MCP servers

Install

Run in a terminal

claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcp

Add to Settings → Connectors → Add custom connector

https://api.llmbenchmark.ai/mcp
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "llmbenchmark": {
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "llmbenchmark": {
      "type": "http",
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "llmbenchmark": {
      "serverUrl": "https://api.llmbenchmark.ai/mcp"
    }
  }
}

Add to Grok Bot desktop app → Connect apps → Add a custom connector

https://api.llmbenchmark.ai/mcp

Enter a Name (for example "LLMBenchmark.ai"), choose Type: Remote HTTPS, paste this as the Server URL and choose Add plugin.

Full Grok Bot setup guide

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "llmbenchmark": {
      "type": "http",
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

About LLMBenchmark.ai

The README says the server answers the question of which model to pick for a given job, using benchmark scores and API list prices for hundreds of language models. Rather than running a language model itself, it maps the task text to a category through a fixed keyword table, filters models by price, context window and open-weights limits, and ranks what remains on that category's leaderboard.

Beyond recommendations, it exposes six read-only tools for leaderboards, side-by-side model comparisons, single-model details, price lookups and category listings. It is reachable as a remote Streamable HTTP endpoint with no key, and the same data is available over a plain REST API with an OpenAPI description.

It suits developers and AI agents that need to choose a model for coding, agentic, reasoning, math, knowledge or web design work and want figures they can cite. The service is limited to 500 requests per IP per UTC day, and the data is refreshed once a day under CC BY 4.0.

Tools this server provides (6)

Taken from the project documentation; the running server may differ.

recommend_model
Describe a task, set a price, context or open-weights limit, and get the top models with scores, rank, prices and a one-line reason.
get_leaderboard
The top models on one category board: overall, coding, agentic, reasoning, math, knowledge or design.
compare_models
Two models side by side on every board where both were measured, with prices and context.
get_model
One model: maker, release date, prices, context window, open weights, and its score and rank on every board.
get_prices
API list prices per million tokens, cheapest output price first, filtered by model or maker.
list_categories
The categories, the board behind each and the task keywords that select it.

What it does

Task-based model recommendations
Describe a job in plain text and receive the top-ranked models that fit your price, context and open-weights limits.
Six read-only tools
Leaderboard, comparison, model detail, price, and category tools are exposed over MCP.
Deterministic category selection
Keyword matching picks the leaderboard without calling a language model, so identical input returns identical output.
Dated data with source links
Answers carry the data date, source name and links to the underlying pages.
REST twin of every tool
The same functions are available under a v1 HTTP API with CORS enabled for GET requests.
Bulk dataset download
The full dataset can be downloaded as JSON for offline or high-volume use.

Good for

  • Choosing a coding model for a refactor within an output price budget
  • Finding the cheapest open-weight model that meets a context window requirement
  • Comparing two models across every benchmark board where both were measured
  • Letting an AI agent cite current benchmark and price data in its answers
  • Pulling LLM price lists by maker for cost planning

Pricing

Free

Free, no key, 500 requests per IP per day.

Prices change. Check llmbenchmark.ai before you buy.

LLMBenchmark.ai FAQ

Does LLMBenchmark.ai require an API key or sign-up?

No. The page and README say it is free, needs no key and requires no sign-up, subject to 500 requests per IP per UTC day.

How often is the data updated?

The site says data refreshes once a day, and every answer includes a data_date field.

What license applies to the data?

The data is licensed CC BY 4.0, so it can be used commercially with credit to LLMBenchmark.ai and a link to the site.

How does recommend_model choose a category?

It counts task keywords for each category, picks the one with the most hits, and falls back to the overall board when no keyword matches.

Is LLMBenchmark.ai free?

Yes. LLMBenchmark.ai is free to use.

How do I add LLMBenchmark.ai to Claude Code?

Run this command in your terminal: claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcp

What are the best alternatives to LLMBenchmark.ai?

The closest alternatives to LLMBenchmark.ai are Hello, AI, BenchLeader, Quelllm MCP, AI Model Analyzer and apipriceindex.com.

Where this comes from

  • Description: written from the project's README, the project's website and its MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers