Skip to content
MCP server

Agent Reliability

Hosted MCP server exposing a sourced knowledge graph on AI agent testing and evaluation

Install
Platforms
API, CLI
Runs
Remotely
Version
0.6.14
Listing updated
today

Similar MCP servers

About Agent Reliability

Agent Reliability is an MCP server for agent benchmarks. It runs as a hosted remote server, so there is nothing to install locally.

The README says the server gives AI clients read-only access to a curated corpus about testing, benchmarking and auditing autonomous agents. Each answer comes back with its cited sources, an evidence tier, a confidence value and a last-verified date, so a client can decide how much weight to give a claim before using it.

The endpoint is a remote streamable-HTTP service at agentreliability.dev/mcp. Clients connect without an API key or account, and the README lists eight tools, including full-text search, a direct question-answering tool, entity and relation lookups, and a source registry. Responses are structured, and the README documents rate limiting of 120 requests per minute per client.

It suits developers and evaluation teams who want an agent to look up eval methods, judge calibration or fault-injection approaches and cite where each point comes from. The corpus covers 38 knowledge objects drawn from 31 registered sources, so it is a focused reference rather than a general search tool.

Install

Run in a terminal

claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'

Add to Settings → Connectors → Add custom connector

https://agentreliability.dev/mcp?via=manifest
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "agent-reliability": {
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "agent-reliability": {
      "type": "http",
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "agent-reliability": {
      "serverUrl": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "agent-reliability": {
      "type": "http",
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Tools this server provides (8)

Taken from the project documentation; the running server may differ.

get_overview
Corpus overview with counts, published tags and freshness.
search
Full-text search over the knowledge graph.
answer
Answer a question from the corpus with sourced claims.
get_entity
Fetch one knowledge object by id with its claims and sources.
get_topic
List knowledge objects carrying a tag.
get_related
Graph neighbours of an object with relation types.
get_sources
Source registry, or sources cited by one object.
get_latest
Most recently verified knowledge objects.

What it does

Sourced claims with confidence
Each returned claim carries its citations, a confidence value, an evidence tier and a verification date.
Full-text corpus search
Search across the knowledge graph with relevance scores and the fields each hit matched.
Direct question answering
An answer tool returns matched claims with sources and reports when the corpus cannot answer.
Graph and source lookups
Fetch objects by id, list topics by tag, follow related objects and inspect the source registry.
No-auth hosted endpoint
Connect over streamable HTTP with no installation, API key or account; access is read-only.

Good for

  • Look up how to tell a genuine evaluation from benchmark contamination
  • Check methods for calibrating an LLM-as-judge before relying on its scores
  • Find which failure modes fault injection and chaos testing can surface in agents
  • Verify the source behind an agent testing claim before citing it

Agent Reliability FAQ

Does the server require an API key?

No. The README states there is no API key, no account and no auth, and that the server is read-only.

What transport does it use?

It is a hosted streamable-HTTP endpoint at https://agentreliability.dev/mcp, and the README also lists it on Smithery.

How many tools does it expose?

The README lists eight tools, each with an output schema.

What is the rate limit?

The README says 120 requests per minute per client, with RateLimit headers returned on responses.

How do I add Agent Reliability to Claude Code?

Run this command in your terminal: claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'

What are the best alternatives to Agent Reliability?

The closest alternatives to Agent Reliability are ScoreIA Open Chamber, First Dollar Test, mizan-ledger, Word Is Bond and ICONIC — MCP Selection Test for MCP publishers.

Where this comes from

  • Description: written from the project's README and its MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers