# Open SEO Crawler

> Open SEO Crawler is an open-source MCP server for web search and scraping. Self-hosted SEO crawler with AI-crawler checks that assistants can drive over MCP. It runs locally on your machine and is started by your AI client.

- **Kind:** MCP server
- **Pricing:** Open source
- **Platforms:** self-hosted, CLI, macOS, Windows, Linux, Web
- **Repository:** https://github.com/puneetindersingh/open-seo-crawler
- **MCP Registry name:** io.github.puneetindersingh/open-seo-crawler
- **Categories:** web search and scraping, marketing and SEO
- **Listing updated:** 2026-11-16

## About

Open SEO Crawler is a local website auditing tool, and the MCP server lets an assistant run it. Through MCP or the bundled osc.py command line, an assistant can start a crawl and follow it, read the summary, filter pages by issue, open a single page with its inlinks, compare two crawls, run the AI crawler check on any URL and export to .xlsx. The crawler itself calls no AI.

After each crawl it requests a sample of pages as Googlebot, Bingbot, Applebot and AI crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot and CCBot, flagging blocks, challenges, redirects, thinner responses and content that only appears after JavaScript runs.

The README describes it as MIT-licensed with no accounts, API keys, cloud or URL limits, with CMS-aware presets for ten CMSs and optional headless rendering. The MCP server ships as an mcpb bundle and runs over stdio.

## Features

- **AI crawler check:** Requests pages as Googlebot, Bingbot and several AI crawlers to spot blocks and thinner versions.
- **Crawl control by MCP:** Start, follow and read crawls from an assistant.
- **Issue filtering:** Filter pages by issue and open one page with its inlinks.
- **Crawl comparison:** Compare two crawls and export results to .xlsx.
- **CMS-aware presets:** Ten CMS presets and optional headless JS rendering.
- **Use with AI brief:** Packs a crawl into a Markdown summary and prompt for any assistant.

## Good for

- Audit a site for technical SEO issues from chat
- Check whether AI search bots can read key pages
- Compare crawls before and after a release
- Run recurring portfolio-wide site audits

## Install

### Claude Code (Terminal)

```sh
claude mcp add open-seo-crawler -- npx -y https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb
```

### Claude Desktop (claude_desktop_config.json)

```json
{
  "mcpServers": {
    "open-seo-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb"
      ]
    }
  }
}
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "open-seo-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb"
      ]
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "open-seo-crawler": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb"
      ]
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "open-seo-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb"
      ]
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "open-seo-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb"
      ]
    }
  }
}
```

## FAQ

### Does it need an account or API key?

No. The README says it runs locally with no accounts, API keys or cloud.

### Does the crawler use AI itself?

No. Your assistant calls the crawler; the crawler calls no AI.

### What licence is it?

The README lists it as MIT.

### Is Open SEO Crawler free?

Yes. Open SEO Crawler is open source, so you can use and self-host it for free.

### How do I add Open SEO Crawler to Claude Code?

Run this command in your terminal: claude mcp add open-seo-crawler -- npx -y https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb

### What are the best alternatives to Open SEO Crawler?

The closest alternatives to Open SEO Crawler are Scrapling MCP Server, Firecrawl MCP Server, Google Ads + Meta Ads + SEO/GEO MCP, SearXNG Search and Calven.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [Google Ads + Meta Ads + SEO/GEO MCP](https://ai-powered.com/mcp/google-ads-meta-ads-seo-geo-mcp): Google Ads + Meta Ads + SEO/GEO MCP is a freemium MCP server for marketing and seo.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [Calven](https://ai-powered.com/mcp/calven): Calven is an MCP server for marketing and seo.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [MCP](https://ai-powered.com/mcp/mcp-mwhffe53): MCP is a freemium MCP server for marketing and seo.

---
Source: https://ai-powered.com/mcp/open-seo-crawler
