# Agent Reliability

> Agent Reliability is an MCP server for agent benchmarks. Hosted MCP server exposing a sourced knowledge graph on AI agent testing and evaluation. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Platforms:** API, CLI
- **Repository:** https://github.com/citarium/agentreliability-mcp
- **MCP Registry name:** dev.agentreliability/agent-reliability
- **Categories:** [agent benchmarks](https://ai-powered.com/categories/agent-benchmarks)
- **Listing updated:** 2026-11-18

## About

The README says the server gives AI clients read-only access to a curated corpus about testing, benchmarking and auditing autonomous agents. Each answer comes back with its cited sources, an evidence tier, a confidence value and a last-verified date, so a client can decide how much weight to give a claim before using it.

The endpoint is a remote streamable-HTTP service at agentreliability.dev/mcp. Clients connect without an API key or account, and the README lists eight tools, including full-text search, a direct question-answering tool, entity and relation lookups, and a source registry. Responses are structured, and the README documents rate limiting of 120 requests per minute per client.

It suits developers and evaluation teams who want an agent to look up eval methods, judge calibration or fault-injection approaches and cite where each point comes from. The corpus covers 38 knowledge objects drawn from 31 registered sources, so it is a focused reference rather than a general search tool.

## Features

- **Sourced claims with confidence:** Each returned claim carries its citations, a confidence value, an evidence tier and a verification date.
- **Full-text corpus search:** Search across the knowledge graph with relevance scores and the fields each hit matched.
- **Direct question answering:** An answer tool returns matched claims with sources and reports when the corpus cannot answer.
- **Graph and source lookups:** Fetch objects by id, list topics by tag, follow related objects and inspect the source registry.
- **No-auth hosted endpoint:** Connect over streamable HTTP with no installation, API key or account; access is read-only.

## Good for

- Look up how to tell a genuine evaluation from benchmark contamination
- Check methods for calibrating an LLM-as-judge before relying on its scores
- Find which failure modes fault injection and chaos testing can surface in agents
- Verify the source behind an agent testing claim before citing it

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://agentreliability.dev/mcp?via=manifest
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "agent-reliability": {
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "agent-reliability": {
      "type": "http",
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "agent-reliability": {
      "serverUrl": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "agent-reliability": {
      "type": "http",
      "url": "https://agentreliability.dev/mcp?via=manifest"
    }
  }
}
```

## Tools

- `get_overview`: Corpus overview with counts, published tags and freshness.
- `search`: Full-text search over the knowledge graph.
- `answer`: Answer a question from the corpus with sourced claims.
- `get_entity`: Fetch one knowledge object by id with its claims and sources.
- `get_topic`: List knowledge objects carrying a tag.
- `get_related`: Graph neighbours of an object with relation types.
- `get_sources`: Source registry, or sources cited by one object.
- `get_latest`: Most recently verified knowledge objects.

## FAQ

### Does the server require an API key?

No. The README states there is no API key, no account and no auth, and that the server is read-only.

### What transport does it use?

It is a hosted streamable-HTTP endpoint at https://agentreliability.dev/mcp, and the README also lists it on Smithery.

### How many tools does it expose?

The README lists eight tools, each with an output schema.

### What is the rate limit?

The README says 120 requests per minute per client, with RateLimit headers returned on responses.

### How do I add Agent Reliability to Claude Code?

Run this command in your terminal: claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'

### What are the best alternatives to Agent Reliability?

The closest alternatives to Agent Reliability are ScoreIA Open Chamber, First Dollar Test, mizan-ledger, Word Is Bond and ICONIC — MCP Selection Test for MCP publishers.

## Where this comes from

- Description: written from the project's README and its MCP Registry entry.
- Install steps: from the official MCP Registry entry.

## Alternatives

- [ScoreIA Open Chamber](https://ai-powered.com/mcp/scoreia-open-chamber): ScoreIA Open Chamber is a free MCP server for agent benchmarks.
- [First Dollar Test](https://ai-powered.com/mcp/first-dollar-test): First Dollar Test is a paid MCP server for agent benchmarks.
- [mizan-ledger](https://ai-powered.com/mcp/mizan-ledger): mizan-ledger is an open-source MCP server for agent benchmarks.
- [Word Is Bond](https://ai-powered.com/mcp/word-is-bond): Word Is Bond is a freemium MCP server for agent benchmarks.
- [ICONIC — MCP Selection Test for MCP publishers](https://ai-powered.com/mcp/iconic-mcp-selection-test-for-mcp-publishers): ICONIC — MCP Selection Test for MCP publishers is a paid MCP server for agent benchmarks.
- [RuleDrift](https://ai-powered.com/mcp/ruledrift): RuleDrift is an open-source MCP server for agent benchmarks.
- [Trialground](https://ai-powered.com/mcp/trialground): Trialground is an MCP server for agent benchmarks.

---
Source: https://ai-powered.com/mcp/agent-reliability
