# LLM Prover

> LLM Prover is a paid MCP server for ai models and llm tooling. Compare models on cost and quality, score outputs and analyse agent runs. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Pricing:** Paid
- **Starting price:** $49 a month
- **Platforms:** Web, API
- **Website:** https://llmprover.pysolvr.com
- **MCP Registry name:** com.pysolvr/llmprover
- **Categories:** AI models and LLM tooling
- **Listing updated:** 2026-11-14

## About

LLM Prover treats model evaluation like testing: measure one model as you would a unit test, or run a whole agent workflow as an integration test. A prompt goes to several models at once and returns quality scores, cost and latency side by side, on your own data rather than a public leaderboard.

Scoring can use a gold-standard answer or custom rubric criteria, and higher plans add compliance scoring. Agent analysis reads a workflow's trace and shows where it wasted money, time or effort. Scheduled benchmarks track quality, cost and latency over time with drift alerts, and you can include private, fine-tuned or self-hosted models through any OpenAI-compatible endpoint.

The MCP server is included on the Pro plan and above. Plans are monthly with usage included and optional top-ups: Starter $49 (up to 5 models per run), Pro $129 (8), Pro Plus $199 (10) and Enterprise $499 (20). Starter has REST API access only.

## Features

- **Side-by-side comparison:** Run one prompt across models and see quality, cost and latency together.
- **Gold standard and rubrics:** Score each response against a reference answer or custom criteria.
- **Agent analysis:** Find where an agent workflow wasted money, time or effort.
- **Scheduled benchmarks:** Track drift over time with alerts, from daily to hourly by plan.
- **Bring your own endpoint:** Benchmark any OpenAI-compatible model alongside hosted ones.

## Good for

- Pick the cheapest model that meets a quality bar
- Score outputs against a rubric without manual review
- Find wasteful steps in an agent run
- Watch a production prompt for quality drift

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http llmprover https://mcp.llmprover.pysolvr.com/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://mcp.llmprover.pysolvr.com/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "llmprover": {
      "url": "https://mcp.llmprover.pysolvr.com/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "llmprover": {
      "type": "http",
      "url": "https://mcp.llmprover.pysolvr.com/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "llmprover": {
      "serverUrl": "https://mcp.llmprover.pysolvr.com/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "llmprover": {
      "type": "http",
      "url": "https://mcp.llmprover.pysolvr.com/mcp"
    }
  }
}
```

## FAQ

### Which plan includes the MCP server?

Pro and above; Starter has REST API access only.

### How many models can I compare?

Up to 5 on Starter, 8 on Pro, 10 on Pro Plus and 20 on Enterprise.

### Can I test my own models?

Yes, through any OpenAI-compatible endpoint.

### What does it cost?

Plans run $49, $129, $199 and $499 a month, with usage included and top-ups available.

### Is LLM Prover free?

No. LLM Prover is a paid product. Paid plans start at $49 a month.

### How do I add LLM Prover to Claude Code?

Run this command in your terminal: claude mcp add --transport http llmprover https://mcp.llmprover.pysolvr.com/mcp

### What are the best alternatives to LLM Prover?

The closest alternatives to LLM Prover are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.

## Alternatives

- [Claude Code Ultimate Guide](https://ai-powered.com/mcp/claude-code-ultimate-guide): Claude Code Ultimate Guide is an open-source MCP server for ai models and llm tooling.
- [signafAIde](https://ai-powered.com/mcp/signafaide): signafAIde is a freemium MCP server for security.
- [Toolkien](https://ai-powered.com/mcp/toolkien): Toolkien is an open-source MCP server for ai models and llm tooling.
- [Lugh](https://ai-powered.com/mcp/lugh): Lugh is an open-source MCP server for ai models and llm tooling.
- [HypePaper](https://ai-powered.com/mcp/hypepaper): HypePaper is a freemium MCP server for ai models and llm tooling.
- [Agent Data Tools: Compact (find, describe, run)](https://ai-powered.com/mcp/agent-data-tools-compact-find-describe-run): Agent Data Tools: Compact (find, describe, run) is a freemium MCP server for developer tools.
- [Angel Agent Guard: pre-purchase check + prompt-injection scan](https://ai-powered.com/mcp/angel-agent-guard-pre-purchase-check-prompt-injection-scan): Angel Agent Guard: pre-purchase check + prompt-injection scan is a freemium MCP server for ai models and llm tooling.
- [Mizan Calibration](https://ai-powered.com/mcp/mizan-calibration): Mizan Calibration is a free MCP server for ai models and llm tooling.

---
Source: https://ai-powered.com/mcp/llm-prover
