# Leaklens

> Leaklens is a freemium MCP server for developer tools. Check SWE-bench tasks for leaked fixes: tier, quoted evidence and fix-commit date. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Pricing:** Freemium
- **Platforms:** API, Web
- **Website:** https://leaklens.grain64.com/docs
- **MCP Registry name:** com.grain64.leaklens/leaklens
- **Categories:** developer tools, AI models and LLM tooling
- **Listing updated:** 2026-11-14

## About

Benchmark scores mean less if a model could have seen the answer. This server looks up SWE-bench Verified and Lite tasks and reports whether the issue or its comments state the fix (tier A) or contain lines of the gold patch (tier C), with the quoted passage and the date of the upstream fix commit to compare against a model's training cutoff.

Two read-only tools answer without a key or sign-up over a Streamable HTTP endpoint. One checks a single task or up to 50 at once, so a results file can be screened for flagged tasks. The other returns a benchmark's flagged counts, measured precision with its interval and a ready-to-cite string.

Every result carries a caveat that a fix dated before a cutoff shows exposure was possible, not that the model trained on it. The same data is available over plain GET endpoints and as a CSV, and a paid tier checks your own benchmark.

## Features

- **Per-task leakage tier:** Each task gets tier A, tier C or none, plus a flag, clear, disputed or pending state.
- **Quoted evidence:** Matching passages are shown with their position in the issue or comments.
- **Fix-commit dates:** The upstream fix commit and its date are returned for cutoff comparison.
- **Batch checks:** Pass 1 to 50 task ids to see which tasks in a run are flagged.
- **Citable summary:** Flagged counts and precision come with a ready-to-cite string.
- **Open API and CSV:** The same data is available over GET endpoints and a CSV export.

## Good for

- Screen a SWE-bench results file for leaked tasks before trusting a score
- Drop flagged tasks from an evaluation run
- Cite leakage rates in an eval report
- Compare fix-commit dates with a model's training cutoff
- Check one GitHub issue or pull request URL

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http leaklens https://leaklens.grain64.com/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://leaklens.grain64.com/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "leaklens": {
      "url": "https://leaklens.grain64.com/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "leaklens": {
      "type": "http",
      "url": "https://leaklens.grain64.com/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "leaklens": {
      "serverUrl": "https://leaklens.grain64.com/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "leaklens": {
      "type": "http",
      "url": "https://leaklens.grain64.com/mcp"
    }
  }
}
```

## Tools

- `check_task_leakage`: Check whether coding tasks from SWE-bench-style benchmarks leaked their fix. Pass one instance ID for the full evidence, or up to 50 IDs in the queries field, as a JSON array of strings or one string separated by commas, spaces or new lines (a results file's task list), to get back which are flagged. Each task returns its tier, state and agreement, countedInHeadline, a one-clause why, a quoted sen
- `benchmark_leakage_summary`: How leaky is a coding benchmark? Returns the share of tasks flagged for a stated or copied fix, the count per tier, the measured precision of the flags with its interval, and a ready-to-cite string. Optionally lists the flagged tasks of one repo (flaggedTasks, the headline's rule: Tier A where both checks agreed, plus Tier C) and the tasks flagged but counted apart (notCountedTasks: Tier B, Tier A

## FAQ

### Do I need a key?

No. The MCP endpoint and lookups for SWE-bench Verified and Lite are free with no sign-up.

### Which benchmarks are covered?

SWE-bench Verified and SWE-bench Lite.

### Does a flag prove a model trained on the fix?

No. A fix commit dated before a cutoff shows exposure was possible, not that training happened.

### Can I check my own benchmark?

Yes, on a free tier of 500 task checks a month, with a Pro plan at €49 a month and a €29 one-off audit pack.

### Is Leaklens free?

Leaklens has a free plan, with paid plans for more usage or features.

### How do I add Leaklens to Claude Code?

Run this command in your terminal: claude mcp add --transport http leaklens https://leaklens.grain64.com/mcp

### What are the best alternatives to Leaklens?

The closest alternatives to Leaklens are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.

## Alternatives

- [REA](https://ai-powered.com/mcp/rea): REA is an open-source MCP server for developer tools.
- [Codebase Memory](https://ai-powered.com/mcp/codebase-memory): Codebase Memory is an open-source MCP server for knowledge and memory.
- [GitHub](https://ai-powered.com/mcp/github-2): GitHub is an open-source MCP server for developer tools.
- [Serena MCP: the IDE for your agent](https://ai-powered.com/mcp/serena-mcp-the-ide-for-your-agent): Serena MCP: the IDE for your agent is an open-source MCP server for developer tools.
- [XERJ](https://ai-powered.com/mcp/xerj): XERJ is an open-source MCP server for knowledge and memory.
- [Ouroboros](https://ai-powered.com/mcp/ouroboros): Ouroboros is an open-source MCP server for developer tools.
- [Claude Code Ultimate Guide](https://ai-powered.com/mcp/claude-code-ultimate-guide): Claude Code Ultimate Guide is an open-source MCP server for ai models and llm tooling.
- [Concord MCP](https://ai-powered.com/mcp/concord-mcp): Concord MCP is a freemium MCP server for developer tools.

---
Source: https://ai-powered.com/mcp/leaklens
