LEEVAR reliability battery
MCP server that grades an AI agent's conversations on 18 reliability tests
- Pricing
- Freemium
- Platforms
- macOS, Windows, Linux
- Runs
- Locally
- Version
- 0.1.0
- Listing updated
- 1 day ago
About LEEVAR reliability battery
LEEVAR reliability battery is a freemium MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.
This server wraps the LEEVAR reliability battery so an MCP client can score how well another AI agent behaved in real conversations. You pass in one to eight conversation samples and get back a written report, with a letter grade only when the evidence supports one.
Its rule is never to guess: a dimension the samples cannot evidence is reported as NOT TESTED, and below 13 of 18 evidenced tests no grade is issued at all. The 18 tests fall into six areas, including citation fabrication, multi-step task completion, output consistency, tool use, long-thread recall and error recovery.
Samples are sent to LEEVAR's hosted service and graded by third-party language models. The README says they are deleted 30 days after the scan completes and that the tool descriptions tell the assistant to ask before sending anything. Strip keys and personal data first.
Install
Run in a terminal
claude mcp add leevar -- npx -y https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpbAdd to claude_desktop_config.json
{
"mcpServers": {
"leevar": {
"command": "npx",
"args": [
"-y",
"https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb"
]
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"leevar": {
"command": "npx",
"args": [
"-y",
"https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb"
]
}
}
}Add to .vscode/mcp.json
{
"servers": {
"leevar": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb"
]
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"leevar": {
"command": "npx",
"args": [
"-y",
"https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb"
]
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"leevar": {
"command": "npx",
"args": [
"-y",
"https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb"
]
}
}
}Tools this server provides (3)
Taken from the project documentation; the running server may differ.
- leevar_scan_transcript
- Sends 1-8 real conversation samples to the hosted battery and returns a scan id.
- leevar_scan_status
- Reads the scan result: status, grade, coverage and written report.
- leevar_battery
- Lists the 18 tests and the failure each one catches.
What it does
- Transcript scanning
- leevar_scan_transcript sends 1-8 conversation samples to the hosted battery and returns a scan id.
- Status and report
- leevar_scan_status returns status, grade if issued, coverage and the written report.
- Battery listing
- leevar_battery lists the 18 tests and the failure each one catches.
- No guessed grades
- Untestable dimensions show as NOT TESTED and under 13 evidenced tests no grade is issued.
- Failure credit
- If a scan fails because of LEEVAR, the free scan is credited back.
Good for
- Check a support bot's conversations for hallucinated policies
- Gate a release on grade and coverage
- Find which reliability dimension an agent breaks first
- Choose which transcripts to sample so the battery is exercised
Pricing
Freemium
5 free scans a month per email; Full Diagnostic + Treatment $99 on the website
Prices change. Check leevar.live before you buy.
LEEVAR reliability battery FAQ
Is it free?
The README says 5 scans a month per email, with no account needed. The website also sells a $99 Full Diagnostic + Treatment.
What happens to my transcripts?
They go to the hosted service, are graded by third-party models and are deleted 30 days after the scan; redact secrets and personal data first.
Can it run fully locally?
The repository's root scanner (deno task scan) grades on your own machine with a model you choose.
What does it need to run?
Node 18 or later; it is installed from the v0.1.0 release, not npm.
Is LEEVAR reliability battery free?
LEEVAR reliability battery has a free plan, with paid plans for more usage or features.
How do I add LEEVAR reliability battery to Claude Code?
Run this command in your terminal: claude mcp add leevar -- npx -y https://github.com/forevercrab321-svg/leevar-battery/releases/download/v0.1.0/leevar-mcp-0.1.0.mcpb
What are the best alternatives to LEEVAR reliability battery?
The closest alternatives to LEEVAR reliability battery are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.
Alternatives to LEEVAR reliability battery

REA
MCP serverCLI and MCP server that lets coding agents reverse engineer apps and binaries
Runs locally- Open source
- 464K GitHub stars
- Developer tools

Codebase Memory
MCP serverLocal MCP server that builds a code knowledge graph for AI coding agents
Runs locally- Open source
- 48.7K GitHub stars
- Knowledge and memory

GitHub
MCP serverOfficial MCP server connecting AI assistants to GitHub repos, issues and workflows
Not checked yet- Remote
- Open source
- 34.5K GitHub stars
- Developer tools

Serena MCP: the IDE for your agent
MCP serverMCP toolkit giving coding agents symbol-level code retrieval and editing
Runs locally- Open source
- 31.5K GitHub stars
- Developer tools

XERJ
MCP serverLocal search engine for AI agents with cited answers over your code and docs
Runs locally- Open source
- 19.2K GitHub stars
- Knowledge and memory

Ouroboros
MCP serverSpec-first layer that pins requirements and verifies results of AI coding agent work
Runs locally- Open source
- 6.4K GitHub stars
- Developer tools