Skip to content
MCP server

Crawlbit

Four free tools to check if AI engines can find, read and recognise a site. No API key.

Runs locally
Install
Pricing
Open source
Runs
Locally
Version
1.0.1
Listing updated
3 months ago

About Crawlbit

Crawlbit is an open-source MCP server. It runs locally on your machine and is started by your AI client.

Install

Run in a terminal

claude mcp add crawlbit -- npx -y crawlbit-mcp

Add to claude_desktop_config.json

{
  "mcpServers": {
    "crawlbit": {
      "command": "npx",
      "args": [
        "-y",
        "crawlbit-mcp"
      ]
    }
  }
}
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "crawlbit": {
      "command": "npx",
      "args": [
        "-y",
        "crawlbit-mcp"
      ]
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "crawlbit": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "crawlbit-mcp"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "crawlbit": {
      "command": "npx",
      "args": [
        "-y",
        "crawlbit-mcp"
      ]
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "crawlbit": {
      "command": "npx",
      "args": [
        "-y",
        "crawlbit-mcp"
      ]
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Configuration

  • CRAWLBIT_BASE_URLOptional. Point the server at another CrawlBit instance. Defaults to https://www.crawlbit.app. No API key or account is needed to use this server.

Pricing

Open source

Crawlbit FAQ

Is Crawlbit free?

Yes. Crawlbit is open source, so you can use and self-host it for free.

How do I add Crawlbit to Claude Code?

Run this command in your terminal: claude mcp add crawlbit -- npx -y crawlbit-mcp

Where this comes from

  • Description: from the official MCP Registry entry.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers