Skip to content
MCP server

LLM Assure

Red-team chatbots, RAG apps and agents with 41 OWASP LLM Top 10 probes

Runs locallyStatus and history
Install
Pricing
Open source
Platforms
CLI, self-hosted
Runs
Locally
Version
0.1.0
Listing updated
today

Similar MCP servers

About LLM Assure

LLM Assure is an open-source MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

LLM Assure comes from the K11 LLM Assurance Toolkit and attacks an application the way a tester would. It sends 41 probes grouped into eight suites, such as prompt injection, system prompt leakage, excessive agency and scope control, mapped to the OWASP Top 10 for LLM Applications (2025). Targets can be an HTTP API, an OpenAI-compatible endpoint, Claude or a Python function.

Detection is deterministic first: canary leaks, PII and secrets, refusals and patterns are checked without any model. Putting a unique canary string in the system prompt lets it spot leaks even when reversed or base64-encoded. An optional Claude judge, enabled with an Anthropic key, handles subtler cases, and judge-only probes are marked needs review otherwise.

The MCP tools let Claude browse probes, run a full assessment, send custom multi-turn prompts and read earlier findings. A separate adaptive mode gives a Claude attacker a fixed query budget. Output includes an HTML report with a score, JUnit XML and a CI gate that fails on a chosen severity. The README's demo bot scores 0 out of 100 before remediation and 99 after.

Install

Run in a terminal

claude mcp add llm-assure --env ANTHROPIC_API_KEY='<ANTHROPIC_API_KEY>' -- uvx k11-llm-assure-mcp

Add to claude_desktop_config.json

{
  "mcpServers": {
    "llm-assure": {
      "command": "uvx",
      "args": [
        "k11-llm-assure-mcp"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "llm-assure": {
      "command": "uvx",
      "args": [
        "k11-llm-assure-mcp"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "llm-assure": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "k11-llm-assure-mcp"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "llm-assure": {
      "command": "uvx",
      "args": [
        "k11-llm-assure-mcp"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "llm-assure": {
      "command": "uvx",
      "args": [
        "k11-llm-assure-mcp"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Configuration

  • ANTHROPIC_API_KEYOptional: enables the Claude LLM judge and adaptive red-teaming

Tools this server provides (5)

Taken from the project documentation; the running server may differ.

list_suites
Browses attack categories.
list_probes
Lists the available probes.
run_assessment
Runs a full audit and writes all reports.
send_probe
Sends a custom prompt with history and returns leak, PII and refusal checks.
get_findings
Reads findings from an earlier assessment.json.

What it does

41 mapped probes
Eight suites aligned to the OWASP LLM Top 10 for 2025.
Deterministic detectors
Canary, PII, secret, refusal and pattern checks are free and reproducible.
Optional Claude judge
Evaluates nuanced failures when an Anthropic key is set.
Adaptive red-teaming
A Claude attacker agent probes the target within a query budget.
CI gate and reports
Produces JUnit XML, an HTML audit report and a fail-on severity threshold.
Custom probe packs
Adds industry-specific probes through pack directories.

Good for

  • Fail a CI build when a chatbot leaks its system prompt
  • Audit a RAG assistant before launch
  • Have Claude hunt for injection variants the static suite missed
  • Produce a client-ready assurance report

Pricing

Open source

LLM Assure FAQ

Does it need an API key?

Not for the quick start or deterministic checks. ANTHROPIC_API_KEY is optional and enables the judge and adaptive mode.

What can it test?

HTTP APIs, OpenAI-compatible endpoints such as vLLM, Ollama or Azure, Claude, and Python functions.

Which standard does it follow?

Probes map to the OWASP Top 10 for LLM Applications 2025, plus a business-risk scope suite.

How do I add it to Claude Code?

Run claude mcp add llm-assure -- uvx k11-llm-assure-mcp.

Is LLM Assure free?

Yes. LLM Assure is open source, so you can use and self-host it for free.

How do I add LLM Assure to Claude Code?

Run this command in your terminal: claude mcp add llm-assure --env ANTHROPIC_API_KEY='<ANTHROPIC_API_KEY>' -- uvx k11-llm-assure-mcp

What are the best alternatives to LLM Assure?

The closest alternatives to LLM Assure are REA, headroom, omniget, next-ai-draw-io and mcp-for-blender.

Where this comes from

  • Description: written from the project's README.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers