Botcard
Cited registry of AI crawler robots.txt tokens and a robots.txt checker
- Pricing
- Free
- Platforms
- API, Web
- Runs
- Remotely
- Version
- 2026-10-18.18
- Listing updated
- 1 day ago
About Botcard
Botcard is a free MCP server for marketing and seo. It runs as a hosted remote server, so there is nothing to install locally.
Botcard tells training, search, person-triggered fetching and open crawling apart for AI crawlers. Its registry holds 125 cards from 24 vendors, each citing the vendor's own page and noting when it was checked, with a field such as whether a disallow is an opt-out, whether robots.txt may not apply, or whether the token is only a control.
Four read-only tools are exposed. A checker takes pasted robots.txt text and a path and returns allow or disallow for each token under RFC 9309, along with the vendor's stance. Other tools list crawlers, fetch one card, and suggest a robots.txt from a policy such as allow search and block training. The service never fetches the host you ask about, and the response reports fetched_target false.
The cards flag details that matter, for example that Google-Extended has no user agent of its own and does not change Search inclusion, that OAI-SearchBot is independent of GPTBot, and that Perplexity-User and Meta-ExternalFetcher may ignore robots.txt. No key is needed.
Install
Run in a terminal
claude mcp add --transport http botcard https://whosthatbot.com/mcpAdd to Settings → Connectors → Add custom connector
https://whosthatbot.com/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"botcard": {
"url": "https://whosthatbot.com/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"botcard": {
"type": "http",
"url": "https://whosthatbot.com/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"botcard": {
"serverUrl": "https://whosthatbot.com/mcp"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"botcard": {
"type": "http",
"url": "https://whosthatbot.com/mcp"
}
}
}Tools this server provides (4)
Taken from the project documentation; the running server may differ.
- check_robots
- Checks pasted robots.txt against crawler tokens for a path.
- list_crawlers
- Lists crawlers in the registry.
- get_crawler
- Returns one crawler's card.
- suggest_robots
- Suggests a robots.txt from a stated policy.
What it does
- Cited crawler registry
- 125 cards across 24 vendors with source citations and check dates.
- robots.txt checker
- Answers allow or disallow per token for a path under RFC 9309.
- Policy-based generator
- Suggests a robots.txt from rules such as allow search, block training.
- Purpose classification
- Separates training, search, user fetch and open crawls.
- Control tokens handled
- Notes tokens like Google-Extended that do not crawl.
- No fetching
- Works only on pasted text and never visits a site.
Good for
- Block AI training crawlers but allow search
- Check whether a robots.txt blocks GPTBot
- Look up what a specific crawler token does
- Generate a robots.txt that follows a stated policy
Pricing
Free
Free, no key
Prices change. Check whosthatbot.com before you buy.
Botcard FAQ
Does it visit my site?
No, it only reads the robots.txt text you provide.
How many crawlers are covered?
125 cards from 24 vendors.
Is a key needed?
No.
Does blocking Google-Extended remove me from Search?
The Botcard entry says it does not change Search inclusion or ranking.
Is Botcard free?
Yes. Botcard is free to use.
How do I add Botcard to Claude Code?
Run this command in your terminal: claude mcp add --transport http botcard https://whosthatbot.com/mcp
What are the best alternatives to Botcard?
The closest alternatives to Botcard are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.
Alternatives to Botcard

REA
MCP serverCLI and MCP server that lets coding agents reverse engineer apps and binaries
Runs locally- Open source
- 464K GitHub stars
- Developer tools

Codebase Memory
MCP serverLocal MCP server that builds a code knowledge graph for AI coding agents
Runs locally- Open source
- 48.7K GitHub stars
- Knowledge and memory

GitHub
MCP serverOfficial MCP server connecting AI assistants to GitHub repos, issues and workflows
Not checked yet- Remote
- Open source
- 34.5K GitHub stars
- Developer tools

Serena MCP: the IDE for your agent
MCP serverMCP toolkit giving coding agents symbol-level code retrieval and editing
Runs locally- Open source
- 31.5K GitHub stars
- Developer tools

XERJ
MCP serverLocal search engine for AI agents with cited answers over your code and docs
Runs locally- Open source
- 19.2K GitHub stars
- Knowledge and memory

Ouroboros
MCP serverSpec-first layer that pins requirements and verifies results of AI coding agent work
Runs locally- Open source
- 6.4K GitHub stars
- Developer tools