Skip to content
MCP server

Doorman

Check which AI bots a website's robots.txt lets in, grouped by purpose

Install
Pricing
Free
Platforms
Web, API
Runs
Remotely
Version
1.2.0
Listing updated
2 days ago

About Doorman

Doorman is a free MCP server for marketing and seo. It runs as a hosted remote server, so there is nothing to install locally.

AI crawlers do different jobs, and Doorman sorts them that way: who can train on your site, who can find and cite it in AI search, and who can read it for a person. For example GPTBot trains models, OAI-SearchBot finds pages for ChatGPT search and ChatGPT-User opens a page because someone asked, so you can keep one out and let the others in.

The MCP tool doorman_check_ai_bot_access takes a URL or pasted robots.txt text. The website also lets you pick what you want and copy a fixed robots.txt that keeps your own rules, drops old AI-bot lines and adds one block, including the shared User-agent: __usage_ai-training opt-out. The site says that token comes from an IETF draft submitted on 30 October 2026, with Microsoft honoring it from 1 February 2027 and Google in early 2027. Google Search, Bing and Apple search rules are never touched.

It reads only a public robots.txt and caches it for a short time. The site is clear that robots.txt is a request, not a lock: firewalls and bot-blocking services are separate, and some bots ignore it.

Install

Run in a terminal

claude mcp add --transport http doorman https://doorman.clawhard.com/mcp

Add to Settings → Connectors → Add custom connector

https://doorman.clawhard.com/mcp
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "doorman": {
      "url": "https://doorman.clawhard.com/mcp"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "doorman": {
      "type": "http",
      "url": "https://doorman.clawhard.com/mcp"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "doorman": {
      "serverUrl": "https://doorman.clawhard.com/mcp"
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "doorman": {
      "type": "http",
      "url": "https://doorman.clawhard.com/mcp"
    }
  }
}

Tools this server provides (1)

Taken from the project documentation; the running server may differ.

doorman_check_ai_bot_access
Checks which AI bots a URL or pasted robots.txt allows

What it does

Purpose grouping
Splits bots into training, AI search and on-demand assistant reads.
Check a site or pasted file
Reads a site's public robots.txt or checks text you supply.
robots.txt builder
Generates a fixed file with the shared AI-training opt-out line.
Search protection
Leaves Googlebot and other search crawlers untouched and warns if you block them.
Live badge
A badge for a README or footer shows AI crawler access from your real robots.txt.

Good for

  • Audit which AI crawlers can train on your site
  • Block training while staying visible in AI search
  • Write an updated robots.txt with an AI opt-out
  • Show visitors your AI crawler policy with a badge

Pricing

Free

No account or key needed

Prices change. Check doorman.clawhard.com before you buy.

Doorman FAQ

Can I block AI training and still appear in ChatGPT search?

Yes. Training and search use different bots, so you can disallow GPTBot, ClaudeBot and Google-Extended and allow OAI-SearchBot, Claude-SearchBot and PerplexityBot.

Does blocking Google-Extended hurt Google ranking?

The site says no; it controls Gemini training and grounding, while Google Search uses Googlebot.

Is robots.txt enough?

It is a request the major AI companies say they honor, but some bots ignore it; hard blocking needs a firewall or bot management.

Do I need an account?

No account, key or login.

Is Doorman free?

Yes. Doorman is free to use.

How do I add Doorman to Claude Code?

Run this command in your terminal: claude mcp add --transport http doorman https://doorman.clawhard.com/mcp

What are the best alternatives to Doorman?

The closest alternatives to Doorman are REA, Google Ads + Meta Ads + SEO/GEO MCP, Agent Roast, Midplane and Calven.

Alternatives to Doorman

  1. REA

    MCP server

    CLI and MCP server that lets coding agents reverse engineer apps and binaries

    Runs locally
    • Open source
    • 464K GitHub stars
    • Developer tools
  2. Hosted MCP connector for managing Google Ads, Meta Ads, GA4 and Search Console from chat

    • 8 tools
    • Remote
    • Freemium
    • 8.1K GitHub stars
    • Marketing and SEO
  3. Agent Roast

    MCP server

    Offline review that flags agent changes which weaken tests, assertions or CI

    Runs locally
    • Open source
    • 4 Hacker News points
    • Developer tools
  4. Midplane

    MCP server

    Postgres gateway for AI agents with table grants, column masking and approval for writes

    Runs locally
    • Open source
    • 5 GitHub stars
    • Databases
  5. Calven

    MCP server

    Product marketing context from Calven delivered into Claude, ChatGPT and other AI tools

    Not checked yet
    • Remote
    • Free trial
    • 3 GitHub stars
    • Marketing and SEO
  6. Leash

    MCP server

    Review what your AI touched, approve risky actions and undo supported changes

    • 1 tool
    • Remote
    • Free
    • 1 GitHub star
    • Security