Skip to content
MCP server

LLM Preflight

Check LLM outputs against your own rules before swapping a model, prompt or provider

Runs locally
Install
Platforms
CLI
Runs
Locally
Version
2.25.8
Listing updated
4 days ago

Similar MCP servers

About LLM Preflight

LLM Preflight is an MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

The core idea is a contract: you describe the JSON shape, routing labels or exact text a feature depends on, then test responses against it. A config can include a response that must pass and a negative control that must fail, so the validator itself gets checked. Exit codes carry the decision: 0 for pass, 1 for fail, 2 for an invalid config and 3 for inconclusive.

Two extras go beyond validation. A source audit scans a repository for literal model IDs that are retired or retiring across Anthropic, OpenAI and selected Gemini models, and suggests a replacement. Dry runs preview provider requests with estimated cost before any call is made. A local mock provider lets you try everything without keys, though a mock run is always inconclusive.

The README is careful about claims: a passing run is evidence for your account, prompts and validators, not production approval. It publishes an observed comparison where 26 model routes ran eight support-routing cases each, with 200 of 208 checks passing. It runs as a CLI, GitHub Action or stdio MCP server and needs Python 3.10 or later, with no third-party runtime dependencies.

Install

Run in a terminal

claude mcp add llm-preflight -- uvx llm-preflight

Add to claude_desktop_config.json

{
  "mcpServers": {
    "llm-preflight": {
      "command": "uvx",
      "args": [
        "llm-preflight"
      ]
    }
  }
}
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "llm-preflight": {
      "command": "uvx",
      "args": [
        "llm-preflight"
      ]
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "llm-preflight": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "llm-preflight"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "llm-preflight": {
      "command": "uvx",
      "args": [
        "llm-preflight"
      ]
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "llm-preflight": {
      "command": "uvx",
      "args": [
        "llm-preflight"
      ]
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

What it does

Output contracts
Define allowed labels, JSON shape or exact output and check replies against them.
Negative controls
Fixtures that must fail prove the validator catches bad replies.
Retired model audit
Scans source for model IDs that are retired or retiring and names replacements.
Dry-run cost preview
Shows provider requests and estimated cost before spending anything.
CI exit codes
Distinct codes for pass, fail, invalid config and inconclusive.
Local mock provider
Try the workflow offline without keys or provider calls.

Good for

  • Gate a model upgrade in CI
  • Find hard-coded retired model IDs in a repo
  • Verify a support-routing prompt only returns allowed labels
  • Compare cost and latency of several providers on one contract
  • Keep evidence for a prompt change review

LLM Preflight FAQ

Do I need an API key to try it?

No. The quick start and the source audit work without a key; live checks need provider credentials.

Does a pass mean it is safe for production?

No. The README says a passing run is evidence for your account, prompts and validators, not production approval.

Which providers does the model audit cover?

Anthropic, OpenAI and selected Gemini models.

What does it need to run?

Python 3.10 or later; the README says there are no third-party runtime dependencies.

How can I use it?

As a CLI, a GitHub Action or a local MCP server.

How do I add LLM Preflight to Claude Code?

Run this command in your terminal: claude mcp add llm-preflight -- uvx llm-preflight

What are the best alternatives to LLM Preflight?

The closest alternatives to LLM Preflight are DecisionLint: Jev/Luna Lint, Promptster, GroundLens, LLM Prover and LLM Assure.

Where this comes from

  • Description: written from the project's README.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers