Skip to content
MCP server

Iris

Score AI agent runs for quality, PII leaks, prompt injection and cost, on your own machine

Runs locallyStatus and history
Install
Pricing
Open source
Platforms
self-hosted, CLI, macOS
Runs
Locally
Version
0.21.0
Listing updated
2 days ago

About Iris

Iris is an open-source MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

Iris takes an agent run and returns a verdict instead of leaving quality to eyeballing a few prompts. Your agent calls log_trace, a host hook forwards each turn, or the iris-eval ingest command loads a trace file, and every run received is scored against 28 built-in rules. PII detection covers 21 patterns such as SSN, card numbers, IBAN, medical record numbers, API keys and seed phrases, and prompt injection checks use 38 patterns. Traces record hierarchical spans, per-tool latency, token usage and cost in USD, with budget thresholds per trace or agent.

Twelve tools cover logging, rule evaluation, an LLM-as-judge and semantic citation verification. The judge and citation check use your own Anthropic or OpenAI key, with a per-evaluation cost cap, and no traffic goes through the vendor. Data stays in local SQLite and a dashboard serves at localhost:6920; npx @iris-eval/mcp-server --demo seeds sample failures.

It needs Node.js 22.13 or later, installs for clients such as Claude Code, Cursor, Windsurf, Zed and Codex, and is MIT licensed. Nothing leaves your machine unless you enable an OpenTelemetry export or the LLM judge.

Install

Run in a terminal

claude mcp add mcp-server --env IRIS_API_KEY='<IRIS_API_KEY>' --env IRIS_ANTHROPIC_API_KEY='<IRIS_ANTHROPIC_API_KEY>' --env IRIS_OPENAI_API_KEY='<IRIS_OPENAI_API_KEY>' --env IRIS_WEBHOOK_SECRET='<IRIS_WEBHOOK_SECRET>' -- npx -y @iris-eval/mcp-server

Add to claude_desktop_config.json

{
  "mcpServers": {
    "mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<IRIS_API_KEY>",
        "IRIS_ANTHROPIC_API_KEY": "<IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<IRIS_OPENAI_API_KEY>",
        "IRIS_WEBHOOK_SECRET": "<IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<IRIS_API_KEY>",
        "IRIS_ANTHROPIC_API_KEY": "<IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<IRIS_OPENAI_API_KEY>",
        "IRIS_WEBHOOK_SECRET": "<IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<IRIS_API_KEY>",
        "IRIS_ANTHROPIC_API_KEY": "<IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<IRIS_OPENAI_API_KEY>",
        "IRIS_WEBHOOK_SECRET": "<IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<IRIS_API_KEY>",
        "IRIS_ANTHROPIC_API_KEY": "<IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<IRIS_OPENAI_API_KEY>",
        "IRIS_WEBHOOK_SECRET": "<IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server"
      ],
      "env": {
        "IRIS_API_KEY": "<IRIS_API_KEY>",
        "IRIS_ANTHROPIC_API_KEY": "<IRIS_ANTHROPIC_API_KEY>",
        "IRIS_OPENAI_API_KEY": "<IRIS_OPENAI_API_KEY>",
        "IRIS_WEBHOOK_SECRET": "<IRIS_WEBHOOK_SECRET>"
      }
    }
  }
}

Configuration

  • IRIS_API_KEYAPI key for HTTP authentication. Required to bind the HTTP transport or the dashboard beyond loopback (0.0.0.0, a LAN address, a container): without it the server refuses to start
  • IRIS_API_KEY_FILEPath to a file whose trimmed contents are the API key (the secret-file pattern Docker and Kubernetes mount). Set this or IRIS_API_KEY, not both; further keys and rotation are security.apiKeys in config.json
  • IRIS_ALLOW_UNAUTHENTICATEDSet to 1 to run a non-loopback bind with NO API key on purpose (lifts the startup refusal; the network is then your boundary)
  • IRIS_DB_PATHSQLite database path
  • IRIS_SQLITE_DRIVERWhich SQLite driver holds the database: native (better-sqlite3, the default) or node (Node's built-in node:sqlite, Node 22.13+). Unset: native, falling back to node when the native module cannot load
  • IRIS_SEARCH_BUDGET_MSHow long one trace search may read before it answers with the matches found so far, in milliseconds (50-60000, default 1000)
  • IRIS_SEARCH_INDEXon (default) or off. off keeps no full-text index of the traces: writes are cheaper, and a search reads the traces within its time budget
  • IRIS_LOG_LEVELLog level: debug, info, warn, error
  • IRIS_HOMEDirectory for the database, custom rules and preferences (default ~/.iris)
  • IRIS_ANTHROPIC_API_KEYEnables evaluate_with_llm_judge and verify_citations with an Anthropic model. Optional: the deterministic rules never need a key
  • IRIS_OPENAI_API_KEYEnables evaluate_with_llm_judge and verify_citations with an OpenAI model. Optional: the deterministic rules never need a key
  • IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVALHard cap on the worst-case cost of one LLM judge call (default 0.25); max_cost_usd can lower it for a call, never raise it
  • IRIS_RELEVANCE_JUDGE_MODELA priced judge model; answers_the_ask then asks it on every evaluation with input, sending that input and output to the model's provider, and gates on its verdict (off by default)
  • IRIS_LLM_JUDGE_DAILY_BUDGET_USDWhat every judge call together may spend per UTC day, per tenant, in USD (default 1): the relevance judge, evaluate_with_llm_judge and verify_citations. Past it no judge call is made
  • IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USDThe name of IRIS_LLM_JUDGE_DAILY_BUDGET_USD before 0.21.0. Read when the new name is unset, and it then limits every judge call
  • IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUESTRelevance judge calls one request may make (default 20)
  • IRIS_RELEVANCE_JUDGE_REDACTon (default) replaces the PII and credentials no_pii flags before the relevance judge sends input and output to its provider; off sends them as they are
  • IRIS_CITATION_ALLOW_FETCHLet verify_citations fetch URLs (off by default); a tool argument can turn it off for a call, never on
  • IRIS_DASHBOARDStart the dashboard alongside the server (same as --dashboard)
  • IRIS_DASHBOARD_PORTDashboard port (default 6920); IRIS_PORT is the MCP HTTP transport port (default 3000)
  • IRIS_PORTMCP HTTP transport port (default 3000); only used with --transport http
  • IRIS_OTEL_ENDPOINTOTLP/HTTP collector URL; when set, every stored trace is also exported as OpenTelemetry spans (best-effort, asynchronous)
  • IRIS_CITATION_DOMAINSComma-separated domain allowlist for verify_citations fetches (only consulted when IRIS_CITATION_ALLOW_FETCH is on); a tool argument can narrow it, never add to it
  • IRIS_CITATION_MAX_COST_USD_TOTALThe most one verify_citations call may spend on judge calls, in USD (default 1); max_cost_usd_total can lower it, never raise it
  • IRIS_TRANSPORTTransport: stdio (default) or http; same as --transport
  • IRIS_HOSTBind host for the HTTP transport (default 127.0.0.1); only used with --transport http
  • IRIS_ALLOWED_ORIGINSComma-separated Origin allowlist for the HTTP transport and dashboard CORS (default http://localhost:*)
  • IRIS_MAX_INPUT_ELEMENTSThe most array elements and object keys one input may hold before it is validated (default 100000): a tool call's arguments, and a dashboard or OTLP JSON request body. A tool call over it gets an error result; a body, 413. OTLP protobuf has its own ceiling, one element per 5 bytes of the request size limit
  • IRIS_MAX_TOOL_INPUT_ELEMENTSThe name of IRIS_MAX_INPUT_ELEMENTS before 0.21.0, read when the new name is unset
  • IRIS_DASHBOARD_HOSTBind host for the dashboard (default 127.0.0.1); same as --dashboard-host
  • IRIS_OTEL_HEADERSComma-separated key=value headers sent with every OTLP export (for example an auth token for your collector)
  • IRIS_OTEL_SERVICE_NAMEservice.name resource attribute on exported spans (default iris-eval)
  • IRIS_OTEL_TIMEOUT_MSTimeout in milliseconds for each OTLP export request
  • IRIS_WEBHOOK_URLThe receiver of the webhook that fires on a moment (docs/webhooks.md); merged over notify.webhook in config.json
  • IRIS_WEBHOOK_SECRETThe signing key of that webhook (any string, or whsec_ + base64); the iris format refuses to run without one

Tools this server provides (3)

Taken from the project documentation; the running server may differ.

log_trace
Captures a full agent run with spans, latency, token usage and cost.
evaluate_with_llm_judge
Scores an output with an LLM judge using your own API key.
verify_citations
Checks that cited sources support the output.

What it does

Rule-based scoring
28 built-in rules plus custom Zod rules for output quality and safety.
PII and injection checks
21 PII patterns and 38 prompt-injection patterns.
Cost tracking
Per-trace cost in USD with budget thresholds.
LLM judge
Optional judging with your own Anthropic or OpenAI key and a cost cap.
Local dashboard
Trace visualization and failures view backed by SQLite.
One-command install
Writes config for ten MCP clients and pins the version.

Good for

  • Catch a PII leak in agent output
  • Flag an agent run that overspent
  • Audit agent traces from a file
  • Verify citations in a generated answer

Pricing

Open source

MIT licensed, self-hosted, no limits; no hosted plan

Prices change. Check iris-eval.com before you buy.

Iris FAQ

Is it free?

Yes. The core is MIT licensed with no usage limits or account, and no hosted version exists today.

What does it need?

Node.js 22.13 or later.

Where is data stored?

Locally in SQLite; nothing leaves your machine unless you enable OpenTelemetry export or the LLM judge.

Does the LLM judge route through the vendor?

No. It calls Anthropic or OpenAI directly with your own key.

Is Iris free?

Yes. Iris is open source, so you can use and self-host it for free.

How do I add Iris to Claude Code?

Run this command in your terminal: claude mcp add mcp-server --env IRIS_API_KEY='<IRIS_API_KEY>' --env IRIS_ANTHROPIC_API_KEY='<IRIS_ANTHROPIC_API_KEY>' --env IRIS_OPENAI_API_KEY='<IRIS_OPENAI_API_KEY>' --env IRIS_WEBHOOK_SECRET='<IRIS_WEBHOOK_SECRET>' -- npx -y @iris-eval/mcp-server

What are the best alternatives to Iris?

The closest alternatives to Iris are REA, Claude Code Ultimate Guide, Agent Roast, Midplane and Leash.

Alternatives to Iris

  1. REA

    MCP server

    CLI and MCP server that lets coding agents reverse engineer apps and binaries

    Runs locally
    • Open source
    • 464K GitHub stars
    • Developer tools
  2. MCP server for searching the Claude Code Ultimate Guide from any MCP client

    Runs locally
    • Open source
    • 6.5K GitHub stars
    • AI models and LLM tooling
  3. Agent Roast

    MCP server

    Offline review that flags agent changes which weaken tests, assertions or CI

    Runs locally
    • Open source
    • 4 Hacker News points
    • Developer tools
  4. Midplane

    MCP server

    Postgres gateway for AI agents with table grants, column masking and approval for writes

    Runs locally
    • Open source
    • 5 GitHub stars
    • Databases
  5. Leash

    MCP server

    Review what your AI touched, approve risky actions and undo supported changes

    • 1 tool
    • Remote
    • Free
    • 1 GitHub star
    • Security