# LLMBenchmark.ai

> LLMBenchmark.ai is a free MCP server for ai models and llm tooling. Free MCP server and REST API that recommends LLMs for a task using benchmark scores. It can run locally on your machine or as a hosted remote server.

- **Kind:** MCP server
- **Pricing:** Free
- **Platforms:** API, CLI
- **Website:** https://llmbenchmark.ai/api
- **Repository:** https://github.com/take-group/llmbenchmark-mcp
- **MCP Registry name:** ai.llmbenchmark/llmbenchmark
- **Categories:** [AI models and LLM tooling](https://ai-powered.com/categories/ai-and-llm)
- **Listing updated:** 2026-11-20

## About

The README says the server answers the question of which model to pick for a given job, using benchmark scores and API list prices for hundreds of language models. Rather than running a language model itself, it maps the task text to a category through a fixed keyword table, filters models by price, context window and open-weights limits, and ranks what remains on that category's leaderboard.

Beyond recommendations, it exposes six read-only tools for leaderboards, side-by-side model comparisons, single-model details, price lookups and category listings. It is reachable as a remote Streamable HTTP endpoint with no key, and the same data is available over a plain REST API with an OpenAPI description.

It suits developers and AI agents that need to choose a model for coding, agentic, reasoning, math, knowledge or web design work and want figures they can cite. The service is limited to 500 requests per IP per UTC day, and the data is refreshed once a day under CC BY 4.0.

## Features

- **Task-based model recommendations:** Describe a job in plain text and receive the top-ranked models that fit your price, context and open-weights limits.
- **Six read-only tools:** Leaderboard, comparison, model detail, price, and category tools are exposed over MCP.
- **Deterministic category selection:** Keyword matching picks the leaderboard without calling a language model, so identical input returns identical output.
- **Dated data with source links:** Answers carry the data date, source name and links to the underlying pages.
- **REST twin of every tool:** The same functions are available under a v1 HTTP API with CORS enabled for GET requests.
- **Bulk dataset download:** The full dataset can be downloaded as JSON for offline or high-volume use.

## Good for

- Choosing a coding model for a refactor within an output price budget
- Finding the cheapest open-weight model that meets a context window requirement
- Comparing two models across every benchmark board where both were measured
- Letting an AI agent cite current benchmark and price data in its answers
- Pulling LLM price lists by maker for cost planning

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://api.llmbenchmark.ai/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "llmbenchmark": {
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "llmbenchmark": {
      "type": "http",
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "llmbenchmark": {
      "serverUrl": "https://api.llmbenchmark.ai/mcp"
    }
  }
}
```

### Grok Bot (Grok Bot desktop app → Connect apps → Add a custom connector)

```sh
https://api.llmbenchmark.ai/mcp
```

Enter a Name (for example "LLMBenchmark.ai"), choose Type: Remote HTTPS, paste this as the Server URL and choose Add plugin.

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "llmbenchmark": {
      "type": "http",
      "url": "https://api.llmbenchmark.ai/mcp"
    }
  }
}
```

## Tools

- `recommend_model`: Describe a task, set a price, context or open-weights limit, and get the top models with scores, rank, prices and a one-line reason.
- `get_leaderboard`: The top models on one category board: overall, coding, agentic, reasoning, math, knowledge or design.
- `compare_models`: Two models side by side on every board where both were measured, with prices and context.
- `get_model`: One model: maker, release date, prices, context window, open weights, and its score and rank on every board.
- `get_prices`: API list prices per million tokens, cheapest output price first, filtered by model or maker.
- `list_categories`: The categories, the board behind each and the task keywords that select it.

## FAQ

### Does LLMBenchmark.ai require an API key or sign-up?

No. The page and README say it is free, needs no key and requires no sign-up, subject to 500 requests per IP per UTC day.

### How often is the data updated?

The site says data refreshes once a day, and every answer includes a data_date field.

### What license applies to the data?

The data is licensed CC BY 4.0, so it can be used commercially with credit to LLMBenchmark.ai and a link to the site.

### How does recommend_model choose a category?

It counts task keywords for each category, picks the one with the most hits, and falls back to the overall board when no keyword matches.

### Is LLMBenchmark.ai free?

Yes. LLMBenchmark.ai is free to use.

### How do I add LLMBenchmark.ai to Claude Code?

Run this command in your terminal: claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcp

### What are the best alternatives to LLMBenchmark.ai?

The closest alternatives to LLMBenchmark.ai are Hello, AI, BenchLeader, Quelllm MCP, AI Model Analyzer and apipriceindex.com.

## Where this comes from

- Description: written from the project's README, the project's website and its MCP Registry entry.
- Install steps: from the official MCP Registry entry.

## Alternatives

- [Hello, AI](https://ai-powered.com/mcp/hello-ai): Hello, AI is an MCP server for ai models and llm tooling.
- [BenchLeader](https://ai-powered.com/mcp/benchleader): BenchLeader is a free MCP server for ai models and llm tooling.
- [Quelllm MCP](https://ai-powered.com/mcp/quelllm-mcp): Quelllm MCP is an open-source MCP server for ai models and llm tooling.
- [AI Model Analyzer](https://ai-powered.com/mcp/ai-model-analyzer): AI Model Analyzer is a free MCP server for ai models and llm tooling.
- [apipriceindex.com](https://ai-powered.com/mcp/apipriceindex-com): apipriceindex.com is a free MCP server for ai models and llm tooling.
- [BenchGecko](https://ai-powered.com/mcp/benchgecko): BenchGecko is a free MCP server for ai models and llm tooling.
- [Model Price Radar](https://ai-powered.com/mcp/model-price-radar): Model Price Radar is an open-source MCP server for ai models and llm tooling.
- [BenchLM Data](https://ai-powered.com/mcp/benchlm-data): BenchLM Data is a freemium MCP server for ai models and llm tooling.

---
Source: https://ai-powered.com/mcp/llmbenchmark-ai
