Agent Reliability
Hosted MCP server exposing a sourced knowledge graph on AI agent testing and evaluation
- Platforms
- API, CLI
- Runs
- Remotely
- Version
- 0.6.14
- Listing updated
- today
Similar MCP servers
About Agent Reliability
Agent Reliability is an MCP server for agent benchmarks. It runs as a hosted remote server, so there is nothing to install locally.
The README says the server gives AI clients read-only access to a curated corpus about testing, benchmarking and auditing autonomous agents. Each answer comes back with its cited sources, an evidence tier, a confidence value and a last-verified date, so a client can decide how much weight to give a claim before using it.
The endpoint is a remote streamable-HTTP service at agentreliability.dev/mcp. Clients connect without an API key or account, and the README lists eight tools, including full-text search, a direct question-answering tool, entity and relation lookups, and a source registry. Responses are structured, and the README documents rate limiting of 120 requests per minute per client.
It suits developers and evaluation teams who want an agent to look up eval methods, judge calibration or fault-injection approaches and cite where each point comes from. The corpus covers 38 knowledge objects drawn from 31 registered sources, so it is a focused reference rather than a general search tool.
Install
Run in a terminal
claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'Add to Settings → Connectors → Add custom connector
https://agentreliability.dev/mcp?via=manifestAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"agent-reliability": {
"url": "https://agentreliability.dev/mcp?via=manifest"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"agent-reliability": {
"type": "http",
"url": "https://agentreliability.dev/mcp?via=manifest"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"agent-reliability": {
"serverUrl": "https://agentreliability.dev/mcp?via=manifest"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"agent-reliability": {
"type": "http",
"url": "https://agentreliability.dev/mcp?via=manifest"
}
}
}Using several servers? Build one config file with the MCP config generator.
Tools this server provides (8)
Taken from the project documentation; the running server may differ.
- get_overview
- Corpus overview with counts, published tags and freshness.
- search
- Full-text search over the knowledge graph.
- answer
- Answer a question from the corpus with sourced claims.
- get_entity
- Fetch one knowledge object by id with its claims and sources.
- get_topic
- List knowledge objects carrying a tag.
- get_related
- Graph neighbours of an object with relation types.
- get_sources
- Source registry, or sources cited by one object.
- get_latest
- Most recently verified knowledge objects.
What it does
- Sourced claims with confidence
- Each returned claim carries its citations, a confidence value, an evidence tier and a verification date.
- Full-text corpus search
- Search across the knowledge graph with relevance scores and the fields each hit matched.
- Direct question answering
- An answer tool returns matched claims with sources and reports when the corpus cannot answer.
- Graph and source lookups
- Fetch objects by id, list topics by tag, follow related objects and inspect the source registry.
- No-auth hosted endpoint
- Connect over streamable HTTP with no installation, API key or account; access is read-only.
Good for
- Look up how to tell a genuine evaluation from benchmark contamination
- Check methods for calibrating an LLM-as-judge before relying on its scores
- Find which failure modes fault injection and chaos testing can surface in agents
- Verify the source behind an agent testing claim before citing it
Agent Reliability FAQ
Does the server require an API key?
No. The README states there is no API key, no account and no auth, and that the server is read-only.
What transport does it use?
It is a hosted streamable-HTTP endpoint at https://agentreliability.dev/mcp, and the README also lists it on Smithery.
How many tools does it expose?
The README lists eight tools, each with an output schema.
What is the rate limit?
The README says 120 requests per minute per client, with RateLimit headers returned on responses.
How do I add Agent Reliability to Claude Code?
Run this command in your terminal: claude mcp add --transport http agent-reliability 'https://agentreliability.dev/mcp?via=manifest'
What are the best alternatives to Agent Reliability?
The closest alternatives to Agent Reliability are ScoreIA Open Chamber, First Dollar Test, mizan-ledger, Word Is Bond and ICONIC — MCP Selection Test for MCP publishers.
Where this comes from
- Description: written from the project's README and its MCP Registry entry.
- Install steps: from the official MCP Registry entry.

