Nerf Watch
Daily scores for Claude, GPT and other models on one private, code-graded test suite
- Pricing
- Free
- Platforms
- API, Web
- Runs
- Remotely
- Version
- 1.0.0
- Listing updated
- 2 days ago
About Nerf Watch
Nerf Watch is a free MCP server for ai models and llm tooling. It runs as a hosted remote server, so there is nothing to install locally.
Nerf Watch tracks whether coding models get worse over time. Each day the same private tests run against Claude models through Claude Code, OpenAI models through Codex, and several others through the OpenCode CLI, with code doing the grading. The questions stay private to limit training on them, and a marker appears on a model's chart when a score jumps more than 10 points between runs.
The MCP server exposes the same public data read-only. One tool returns latest scores with previous runs, changes, 7-day baselines and trends, including model ids. The other takes a model id and returns its retained history, oldest first, with the suite version for each point. Missing days are missing rather than zero, and comparisons are made within one suite version. Test questions and grader details are not returned.
The README cautions that a lower score is a lower score on this suite, not proof that a provider changed a model, since runs are noisy. Both tools share a 300-second cache, and shown data needs a Data: tokenusage.dev credit. The service is unofficial.
Install
Run in a terminal
claude mcp add --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatchAdd to Settings → Connectors → Add custom connector
https://tokenusage.dev/mcp/nerfwatchAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"nerfwatch": {
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"nerfwatch": {
"type": "http",
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"nerfwatch": {
"serverUrl": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"nerfwatch": {
"type": "http",
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Tools this server provides (2)
Taken from the project documentation; the running server may differ.
- get_nerfwatch_scores
- Returns the latest public scores, previous runs, changes, baselines and trends with model ids.
- get_nerfwatch_history
- Returns one model's retained score history, oldest first, with suite versions.
What it does
- Latest scores
- Returns each model's current score, change from the last run and 7-day baseline.
- Model history
- Returns one model's retained scores, oldest first, with suite version.
- Trend labels
- Scores within 2 points of the baseline read as steady.
- Private suite
- Test questions stay hidden to reduce training on them.
- No key needed
- Hosted, read-only, no account.
- Speed probe
- The site also shows daily response times by model.
Good for
- Ask whether a model's quality dipped this week
- Compare Claude and GPT models on the same suite
- Chart a model's score history in a dashboard
- Check if a score change is within the usual noise
Pricing
Free
Free, read-only; credit link required for public display
Prices change. Check tokenusage.dev before you buy.
Nerf Watch FAQ
Does a lower score mean a model was changed?
No. The README says it is only a lower score on this suite, as runs are noisy.
What do I need to use it?
No key or account.
How often is it updated?
Tests run once a day, and responses are cached for 300 seconds.
Can I show the data publicly?
Yes, with a visible Data: tokenusage.dev credit linking to the Nerf Watch page.
Is Nerf Watch free?
Yes. Nerf Watch is free to use.
How do I add Nerf Watch to Claude Code?
Run this command in your terminal: claude mcp add --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatch
What are the best alternatives to Nerf Watch?
The closest alternatives to Nerf Watch are Claude Code Ultimate Guide, signafAIde, Toolkien, Lugh and HypePaper.
Alternatives to Nerf Watch

Claude Code Ultimate Guide
MCP serverMCP server for searching the Claude Code Ultimate Guide from any MCP client
Runs locally- Open source
- 6.5K GitHub stars
- AI models and LLM tooling

signafAIde
MCP serverLocal MCP server to set up, verify and check coverage of signed agent action records
Runs locally- 5 tools
- Freemium
- Security

Toolkien
MCP serverOne MCP gateway for all your MCP servers, with code mode, permissions and audit
Not checked yet- 3 tools
- Remote
- Open source
- AI models and LLM tooling
Lugh
MCP serverWrite, check and dry-run Lugh agent programs with policy, budgets and approvals
Runs locally- 5 tools
- Open source
- AI models and LLM tooling
HypePaper
MCP serverResearch papers, benchmarks, citations and topic experts for AI assistants
- 58 tools
- Remote
- Freemium
- AI models and LLM tooling

One compact entry point that finds, describes and runs hundreds of pay-per-call data tools
- 3 tools
- Remote
- Freemium
- Developer tools