Skip to content
MCP server

RAGScore

Generate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.

Runs locallyStatus and history
Install
Pricing
Open source
Runs
Locally
Version
0.8.6
Listing updated
5 days ago

About RAGScore

RAGScore is an open-source MCP server. It runs locally on your machine and is started by your AI client.

Install

Run in a terminal

claude mcp add ragscore --env OPENAI_API_KEY='<OPENAI_API_KEY>' --env ANTHROPIC_API_KEY='<ANTHROPIC_API_KEY>' -- uvx ragscore

Add to claude_desktop_config.json

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ],
      "env": {
        "OPENAI_API_KEY": "<OPENAI_API_KEY>",
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ],
      "env": {
        "OPENAI_API_KEY": "<OPENAI_API_KEY>",
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "ragscore": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "ragscore"
      ],
      "env": {
        "OPENAI_API_KEY": "<OPENAI_API_KEY>",
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ],
      "env": {
        "OPENAI_API_KEY": "<OPENAI_API_KEY>",
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ],
      "env": {
        "OPENAI_API_KEY": "<OPENAI_API_KEY>",
        "ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>"
      }
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Configuration

  • OPENAI_API_KEYOpenAI API key (if using OpenAI provider)
  • ANTHROPIC_API_KEYAnthropic API key (if using Anthropic provider)

Pricing

Open source

RAGScore FAQ

Is RAGScore free?

Yes. RAGScore is open source, so you can use and self-host it for free.

How do I add RAGScore to Claude Code?

Run this command in your terminal: claude mcp add ragscore --env OPENAI_API_KEY='<OPENAI_API_KEY>' --env ANTHROPIC_API_KEY='<ANTHROPIC_API_KEY>' -- uvx ragscore

Where this comes from

  • Description: from the official MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers