Leaklens
Check SWE-bench tasks for leaked fixes: tier, quoted evidence and fix-commit date
Answered our latest tool-list check 21 hours ago with 2 tools. It will be marked verified after 3 successful checks in 7 days.
- Pricing
- Freemium
- Platforms
- API, Web
- Runs
- Remotely
- Version
- 1.0.0
- Listing updated
- 3 days ago
About Leaklens
Leaklens is a freemium MCP server for developer tools. It runs as a hosted remote server, so there is nothing to install locally.
Benchmark scores mean less if a model could have seen the answer. This server looks up SWE-bench Verified and Lite tasks and reports whether the issue or its comments state the fix (tier A) or contain lines of the gold patch (tier C), with the quoted passage and the date of the upstream fix commit to compare against a model's training cutoff.
Two read-only tools answer without a key or sign-up over a Streamable HTTP endpoint. One checks a single task or up to 50 at once, so a results file can be screened for flagged tasks. The other returns a benchmark's flagged counts, measured precision with its interval and a ready-to-cite string.
Every result carries a caveat that a fix dated before a cutoff shows exposure was possible, not that the model trained on it. The same data is available over plain GET endpoints and as a CSV, and a paid tier checks your own benchmark.
Install
Run in a terminal
claude mcp add --transport http leaklens https://leaklens.grain64.com/mcpAdd to Settings → Connectors → Add custom connector
https://leaklens.grain64.com/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"leaklens": {
"url": "https://leaklens.grain64.com/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"leaklens": {
"type": "http",
"url": "https://leaklens.grain64.com/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"leaklens": {
"serverUrl": "https://leaklens.grain64.com/mcp"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"leaklens": {
"type": "http",
"url": "https://leaklens.grain64.com/mcp"
}
}
}Tools this server provides (2)
Read directly from the server.
- check_task_leakage
- Check whether coding tasks from SWE-bench-style benchmarks leaked their fix. Pass one instance ID for the full evidence, or up to 50 IDs in the queries field, as a JSON array of strings or one string separated by commas, spaces or new lines (a results file's task list), to get back which are flagged. Each task returns its tier, state and agreement, countedInHeadline, a one-clause why, a quoted sen
- benchmark_leakage_summary
- How leaky is a coding benchmark? Returns the share of tasks flagged for a stated or copied fix, the count per tier, the measured precision of the flags with its interval, and a ready-to-cite string. Optionally lists the flagged tasks of one repo (flaggedTasks, the headline's rule: Tier A where both checks agreed, plus Tier C) and the tasks flagged but counted apart (notCountedTasks: Tier B, Tier A
What it does
- Per-task leakage tier
- Each task gets tier A, tier C or none, plus a flag, clear, disputed or pending state.
- Quoted evidence
- Matching passages are shown with their position in the issue or comments.
- Fix-commit dates
- The upstream fix commit and its date are returned for cutoff comparison.
- Batch checks
- Pass 1 to 50 task ids to see which tasks in a run are flagged.
- Citable summary
- Flagged counts and precision come with a ready-to-cite string.
- Open API and CSV
- The same data is available over GET endpoints and a CSV export.
Good for
- Screen a SWE-bench results file for leaked tasks before trusting a score
- Drop flagged tasks from an evaluation run
- Cite leakage rates in an eval report
- Compare fix-commit dates with a model's training cutoff
- Check one GitHub issue or pull request URL
Pricing
Freemium
Public lookups free. Own-benchmark checks: free 500 checks/month, Pro €49/month, audit pack €29 once.
Prices change. Check leaklens.grain64.com before you buy.
Leaklens FAQ
Do I need a key?
No. The MCP endpoint and lookups for SWE-bench Verified and Lite are free with no sign-up.
Which benchmarks are covered?
SWE-bench Verified and SWE-bench Lite.
Does a flag prove a model trained on the fix?
No. A fix commit dated before a cutoff shows exposure was possible, not that training happened.
Can I check my own benchmark?
Yes, on a free tier of 500 task checks a month, with a Pro plan at €49 a month and a €29 one-off audit pack.
Is Leaklens free?
Leaklens has a free plan, with paid plans for more usage or features.
How do I add Leaklens to Claude Code?
Run this command in your terminal: claude mcp add --transport http leaklens https://leaklens.grain64.com/mcp
What are the best alternatives to Leaklens?
The closest alternatives to Leaklens are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.
Alternatives to Leaklens

REA
MCP serverCLI and MCP server that lets coding agents reverse engineer apps and binaries
Runs locally- Open source
- 464K GitHub stars
- Developer tools

Codebase Memory
MCP serverLocal MCP server that builds a code knowledge graph for AI coding agents
Runs locally- Open source
- 48.7K GitHub stars
- Knowledge and memory

GitHub
MCP serverOfficial MCP server connecting AI assistants to GitHub repos, issues and workflows
Not checked yet- Remote
- Open source
- 34.5K GitHub stars
- Developer tools

Serena MCP: the IDE for your agent
MCP serverMCP toolkit giving coding agents symbol-level code retrieval and editing
Runs locally- Open source
- 31.5K GitHub stars
- Developer tools

XERJ
MCP serverLocal search engine for AI agents with cited answers over your code and docs
Runs locally- Open source
- 19.2K GitHub stars
- Knowledge and memory

Ouroboros
MCP serverSpec-first layer that pins requirements and verifies results of AI coding agent work
Runs locally- Open source
- 6.4K GitHub stars
- Developer tools