# Ai Crawler Policy

> Ai Crawler Policy is an MCP server for web search and scraping. Cloudflare Workers MCP server: ai-crawler-policy. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Repository:** https://github.com/lazymac2x/ai-crawler-policy-api
- **MCP Registry name:** io.github.lazymac2x/ai-crawler-policy
- **Categories:** web search and scraping
- **Listing updated:** 2026-11-13

## About

Cloudflare Workers MCP server: ai-crawler-policy

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http ai-crawler-policy https://api.lazy-mac.com/ai-crawler-policy/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://api.lazy-mac.com/ai-crawler-policy/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "ai-crawler-policy": {
      "url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "ai-crawler-policy": {
      "type": "http",
      "url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "ai-crawler-policy": {
      "serverUrl": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "ai-crawler-policy": {
      "type": "http",
      "url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}
```

## FAQ

### How do I add Ai Crawler Policy to Claude Code?

Run this command in your terminal: claude mcp add --transport http ai-crawler-policy https://api.lazy-mac.com/ai-crawler-policy/mcp

### What are the best alternatives to Ai Crawler Policy?

The closest alternatives to Ai Crawler Policy are ActableSite AI Crawler Monitor, Firecrawl Alternative Stealth MCP, AI Crawler Checker — Robots.txt Policy for Six Bots and NM Signals: AI crawler audit.

## Where this comes from

- Description: from the official MCP Registry entry.
- Install steps: from the official MCP Registry entry.

## Alternatives

- [ActableSite AI Crawler Monitor](https://ai-powered.com/mcp/actablesite-ai-crawler-monitor): ActableSite AI Crawler Monitor is an MCP server for web search and scraping.
- [Firecrawl Alternative Stealth MCP](https://ai-powered.com/mcp/firecrawl-alternative-stealth-mcp): Firecrawl Alternative Stealth MCP is an MCP server for web search and scraping.
- [AI Crawler Checker — Robots.txt Policy for Six Bots](https://ai-powered.com/mcp/ai-crawler-checker-robots-txt-policy-for-six-bots): AI Crawler Checker — Robots.txt Policy for Six Bots is an MCP server for web search and scraping.
- [NM Signals: AI crawler audit](https://ai-powered.com/mcp/nm-signals-ai-crawler-audit): NM Signals: AI crawler audit is an MCP server for web search and scraping.

---
Source: https://ai-powered.com/mcp/ai-crawler-policy
