Skip to content
MCP server

falsepass

Test RL environment graders for wrong answers they pay and right ones they reject

Runs locallyStatus and history
Install
Pricing
Open source
Platforms
CLI, self-hosted
Runs
Locally
Version
0.15.0
Listing updated
3 days ago

About falsepass

falsepass is an open-source MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

A grader in an RL environment is effectively an assertion, and falsepass tests it like one: it calls the environment's own reward code on completions whose correct verdict is already known and reports every disagreement. Wrong answers that get paid are false passes, and right answers that are rejected are false fails. No model, rollouts or GPU are involved.

Probes include the reference answer with harmless variations; empty, echoed, every-option, hedged and wrong answers; malformed output; and repeats, plus a mutation search and a static scan of the reward source. Each failing probe becomes a stand-alone repro. The README describes one public environment where echoing the question back earned full reward because the grader only checked that the right answer appeared somewhere in the reply.

It works with verifiers (Environments Hub), OpenEnv, TRL reward functions, Inspect AI and HUD, and every audit runs in its own throw-away virtualenv (uv and Python 3.11+ needed). A GitHub Action can fail a build when a known-bad answer earns reward, and the MCP server exposes scan, init, audit, results, repro and discover.

Install

Run in a terminal

claude mcp add falsepass -- uvx falsepass

Add to claude_desktop_config.json

{
  "mcpServers": {
    "falsepass": {
      "command": "uvx",
      "args": [
        "falsepass"
      ]
    }
  }
}
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "falsepass": {
      "command": "uvx",
      "args": [
        "falsepass"
      ]
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "falsepass": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "falsepass"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "falsepass": {
      "command": "uvx",
      "args": [
        "falsepass"
      ]
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "falsepass": {
      "command": "uvx",
      "args": [
        "falsepass"
      ]
    }
  }
}

Tools this server provides (6)

Taken from the project documentation; the running server may differ.

scan
Static scan of reward code.
init
Sets up an environment for auditing.
audit
Probes the grader.
results
Returns headline numbers and failing probes.
repro
Creates a stand-alone repro for a probe.
discover
Finds environments to audit.

What it does

False pass and fail probes
Feeds graders completions with known verdicts and lists every mismatch.
Mutation and static scan
Searches variations and points at suspicious reward-code lines.
Stand-alone repros
Turns each failing probe into a reproducible script.
Framework coverage
Handles verifiers, OpenEnv, TRL, Inspect AI and HUD environments.
CI gate
A GitHub Action fails builds when a bad answer earns reward.
Agent-ready output
Non-interactive commands with JSON results and exit codes.

Good for

  • Audit an RL environment before training on it
  • Check that a reward function does not pay empty or echoed answers
  • Gate pull requests that change a grader
  • Have a coding agent produce reproduced leads for grader defects

Pricing

Open source

Free Python package

Prices change. Check super-eric.github.io before you buy.

falsepass FAQ

Does it need a model or GPU?

No. It runs the environment's own reward code on known completions.

What can it test?

verifiers, OpenEnv, TRL reward functions, Inspect AI and HUD graders.

How do I try it?

Run uvx falsepass demo, which audits a bundled toy environment.

What are the requirements?

uv on the PATH and Python 3.11 or newer.

Is falsepass free?

Yes. falsepass is open source, so you can use and self-host it for free.

How do I add falsepass to Claude Code?

Run this command in your terminal: claude mcp add falsepass -- uvx falsepass

What are the best alternatives to falsepass?

The closest alternatives to falsepass are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.

Alternatives to falsepass

  1. REA

    MCP server

    CLI and MCP server that lets coding agents reverse engineer apps and binaries

    Runs locally
    • Open source
    • 464K GitHub stars
    • Developer tools
  2. Codebase Memory

    MCP server

    Local MCP server that builds a code knowledge graph for AI coding agents

    Runs locally
    • Open source
    • 48.7K GitHub stars
    • Knowledge and memory
  3. GitHub

    MCP server

    Official MCP server connecting AI assistants to GitHub repos, issues and workflows

    Not checked yet
    • Remote
    • Open source
    • 34.5K GitHub stars
    • Developer tools
  4. MCP toolkit giving coding agents symbol-level code retrieval and editing

    Runs locally
    • Open source
    • 31.5K GitHub stars
    • Developer tools
  5. XERJ

    MCP server

    Local search engine for AI agents with cited answers over your code and docs

    Runs locally
    • Open source
    • 19.2K GitHub stars
    • Knowledge and memory
  6. Ouroboros

    MCP server

    Spec-first layer that pins requirements and verifies results of AI coding agent work

    Runs locally
    • Open source
    • 6.4K GitHub stars
    • Developer tools