Skip to content
MCP server

RAG Documentation Scraper

Crawl any documentation site and return heading-bounded, LLM-ready chunks with source…

Runs locallyStatus and history
Install
Pricing
Open source
Runs
Locally
Version
1.0.0
Listing updated
4 days ago

About RAG Documentation Scraper

RAG Documentation Scraper is an open-source MCP server for web search and scraping. It runs locally on your machine and is started by your AI client.

Crawl any documentation site and return heading-bounded, LLM-ready chunks with source metadata.

Install

Run in a terminal

claude mcp add rag-docs-scraper --env APIFY_TOKEN='<APIFY_TOKEN>' -- npx -y rag-docs-scraper-mcp-server

Add to claude_desktop_config.json

{
  "mcpServers": {
    "rag-docs-scraper": {
      "command": "npx",
      "args": [
        "-y",
        "rag-docs-scraper-mcp-server"
      ],
      "env": {
        "APIFY_TOKEN": "<APIFY_TOKEN>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "rag-docs-scraper": {
      "command": "npx",
      "args": [
        "-y",
        "rag-docs-scraper-mcp-server"
      ],
      "env": {
        "APIFY_TOKEN": "<APIFY_TOKEN>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "rag-docs-scraper": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "rag-docs-scraper-mcp-server"
      ],
      "env": {
        "APIFY_TOKEN": "<APIFY_TOKEN>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "rag-docs-scraper": {
      "command": "npx",
      "args": [
        "-y",
        "rag-docs-scraper-mcp-server"
      ],
      "env": {
        "APIFY_TOKEN": "<APIFY_TOKEN>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "rag-docs-scraper": {
      "command": "npx",
      "args": [
        "-y",
        "rag-docs-scraper-mcp-server"
      ],
      "env": {
        "APIFY_TOKEN": "<APIFY_TOKEN>"
      }
    }
  }
}

Configuration

  • APIFY_TOKENYour Apify API token — https://console.apify.com/account/integrationsRequired

Pricing

Open source

RAG Documentation Scraper FAQ

Is RAG Documentation Scraper free?

Yes. RAG Documentation Scraper is open source, so you can use and self-host it for free.

How do I add RAG Documentation Scraper to Claude Code?

Run this command in your terminal: claude mcp add rag-docs-scraper --env APIFY_TOKEN='<APIFY_TOKEN>' -- npx -y rag-docs-scraper-mcp-server

What are the best alternatives to RAG Documentation Scraper?

The closest alternatives to RAG Documentation Scraper are Scrapling MCP Server, Codebase Memory, XERJ, Firecrawl MCP Server and Claude Code Ultimate Guide.

Alternatives to RAG Documentation Scraper

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Codebase Memory

    MCP server

    Local MCP server that builds a code knowledge graph for AI coding agents

    Runs locally
    • Open source
    • 48.7K GitHub stars
    • Knowledge and memory
  3. XERJ

    MCP server

    Local search engine for AI agents with cited answers over your code and docs

    Runs locally
    • Open source
    • 19.2K GitHub stars
    • Knowledge and memory
  4. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  5. MCP server for searching the Claude Code Ultimate Guide from any MCP client

    Runs locally
    • Open source
    • 6.5K GitHub stars
    • AI models and LLM tooling
  6. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping