# WebCrawlerAPI

> WebCrawlerAPI is a paid MCP server for web search and scraping. Hosted MCP server that turns any public web page into clean Markdown for an assistant. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Pricing:** Paid
- **Starting price:** $99 a month
- **Platforms:** API, Web
- **Website:** https://webcrawlerapi.com/docs/sdk/remote-mcp
- **Repository:** https://github.com/webcrawlerapi/mcp-server
- **MCP Registry name:** com.webcrawlerapi/mcp-server
- **Categories:** web search and scraping
- **Listing updated:** 2026-11-14

## About

A single read-only tool, get_markdown, fetches a public page and returns its main content as Markdown. JavaScript-rendered pages work, and navigation, ads and boilerplate are stripped. The input is one URL of up to 2048 characters.

The server is hosted, so there is nothing to install. You add its URL to Claude Code, Claude.ai or Desktop, Cursor, VS Code or ChatGPT and sign in with a WebCrawlerAPI account through OAuth 2.1; no API key is pasted. Tokens are tied to the server URL and to the organization you approve, and every call is billed to that organization.

Failures come back as short messages, for example an empty balance, a page blocked by robots.txt, an unreachable host, a protected page or one that is too large. Rate limits are 120 requests a minute per organization and 300 per IP, with request bodies capped at 256 KB.

## Features

- **Page to Markdown:** Reads any public http or https page and returns its main content.
- **JavaScript rendering:** Pages that render client-side are supported.
- **OAuth sign-in:** Authenticate with your account; no API key to paste.
- **Clear error messages:** Failures such as blocked pages or empty balance return short explanations.
- **Read-only tool:** The tool is marked read-only and idempotent.
- **Many clients:** Setup steps for Claude, Cursor, VS Code, ChatGPT and MCP Inspector.

## Good for

- Pull documentation pages into a chat for summarising
- Gather article text for a knowledge base
- Read product pages without ads and navigation noise
- Let an assistant check a live page before answering

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http mcp-server https://mcp.webcrawlerapi.com/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://mcp.webcrawlerapi.com/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "mcp-server": {
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "mcp-server": {
      "type": "http",
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "mcp-server": {
      "serverUrl": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "mcp-server": {
      "type": "http",
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

## Tools

- `get_markdown`: Fetches a public web page and returns its main content as Markdown.

## FAQ

### Do I need an API key?

No. Sign-in is OAuth; you approve access for one of your organizations.

### What does a call cost?

Calls are billed to your organization. The pricing page lists pay as you go from $0.002 a page and $99 a month for Professional.

### What are the limits?

120 requests per minute per organization, 300 per minute per IP, and 256 KB request bodies.

### Which pages can it read?

Public http or https pages. Sites that block the request or robots.txt return an error.

### Is WebCrawlerAPI free?

No. WebCrawlerAPI is a paid product. Paid plans start at $99 a month.

### How do I add WebCrawlerAPI to Claude Code?

Run this command in your terminal: claude mcp add --transport http mcp-server https://mcp.webcrawlerapi.com/mcp

### What are the best alternatives to WebCrawlerAPI?

The closest alternatives to WebCrawlerAPI are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [Shopify MCP Server (any store)](https://ai-powered.com/mcp/shopify-mcp-server-any-store): Shopify MCP Server (any store) is a paid MCP server for e-commerce.
- [Chanroll: YouTube Channel Video Scraper (Official API)](https://ai-powered.com/mcp/chanroll-youtube-channel-video-scraper-official-api): Chanroll: YouTube Channel Video Scraper (Official API) is a paid MCP server for design and media.
- [Vidvett: YouTube Video Details Scraper (Official API)](https://ai-powered.com/mcp/vidvett-youtube-video-details-scraper-official-api): Vidvett: YouTube Video Details Scraper (Official API) is a paid MCP server for design and media.

---
Source: https://ai-powered.com/mcp/webcrawlerapi
