# ActableSite AI Crawler Monitor

> ActableSite AI Crawler Monitor is an MCP server for web search and scraping. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Website:** https://actablesite.com/mcp-server
- **Repository:** https://github.com/unitedideas/actablesite-mcp
- **MCP Registry name:** com.actablesite/ai-crawler-robots-monitor
- **Categories:** [web search and scraping](https://ai-powered.com/categories/web-search-and-scraping)
- **Listing updated:** 2026-07-13

## About

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http ai-crawler-robots-monitor https://actablesite.com/mcp/
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://actablesite.com/mcp/
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "ai-crawler-robots-monitor": {
      "url": "https://actablesite.com/mcp/"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "ai-crawler-robots-monitor": {
      "type": "http",
      "url": "https://actablesite.com/mcp/"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "ai-crawler-robots-monitor": {
      "serverUrl": "https://actablesite.com/mcp/"
    }
  }
}
```

### Grok Bot (Grok Bot desktop app → Connect apps → Add a custom connector)

```sh
https://actablesite.com/mcp/
```

Enter a Name (for example "ActableSite AI Crawler Monitor"), choose Type: Remote HTTPS, paste this as the Server URL and choose Add plugin.

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "ai-crawler-robots-monitor": {
      "type": "http",
      "url": "https://actablesite.com/mcp/"
    }
  }
}
```

## FAQ

### How do I add ActableSite AI Crawler Monitor to Claude Code?

Run this command in your terminal: claude mcp add --transport http ai-crawler-robots-monitor https://actablesite.com/mcp/

### What are the best alternatives to ActableSite AI Crawler Monitor?

The closest alternatives to ActableSite AI Crawler Monitor are Crawlsight, Atalaya AI crawler check, Robots.txt & AI Crawler Checker, Site SEO & AI Crawler Audit and Ai Visibility.

## Where this comes from

- Description: from the official MCP Registry entry.
- Install steps: from the official MCP Registry entry.

## Alternatives

- [Crawlsight](https://ai-powered.com/mcp/crawlsight): Crawlsight is an open-source MCP server for marketing and seo.
- [Atalaya AI crawler check](https://ai-powered.com/mcp/atalaya-ai-crawler-check): Atalaya AI crawler check is a free MCP server for web search and scraping.
- [Robots.txt & AI Crawler Checker](https://ai-powered.com/mcp/robots-txt-and-ai-crawler-checker): Robots.txt & AI Crawler Checker is a freemium MCP server for web search and scraping.
- [Site SEO & AI Crawler Audit](https://ai-powered.com/mcp/site-seo-and-ai-crawler-audit): Site SEO & AI Crawler Audit is a free MCP server for web search and scraping.
- [Ai Visibility](https://ai-powered.com/mcp/ai-visibility-2): Ai Visibility is a free MCP server for marketing and seo.
- [Crawl Readiness](https://ai-powered.com/mcp/crawl-readiness): Crawl Readiness is a freemium MCP server for web search and scraping.

---
Source: https://ai-powered.com/mcp/actablesite-ai-crawler-monitor
