WebCrawlerAPI
Hosted MCP server that turns any public web page into clean Markdown for an assistant
- Pricing
- Paid
- Platforms
- API, Web
- Runs
- Remotely
- Version
- 0.1.3
- Listing updated
- 3 days ago
About WebCrawlerAPI
WebCrawlerAPI is a paid MCP server for web search and scraping. It runs as a hosted remote server, so there is nothing to install locally.
A single read-only tool, get_markdown, fetches a public page and returns its main content as Markdown. JavaScript-rendered pages work, and navigation, ads and boilerplate are stripped. The input is one URL of up to 2048 characters.
The server is hosted, so there is nothing to install. You add its URL to Claude Code, Claude.ai or Desktop, Cursor, VS Code or ChatGPT and sign in with a WebCrawlerAPI account through OAuth 2.1; no API key is pasted. Tokens are tied to the server URL and to the organization you approve, and every call is billed to that organization.
Failures come back as short messages, for example an empty balance, a page blocked by robots.txt, an unreachable host, a protected page or one that is too large. Rate limits are 120 requests a minute per organization and 300 per IP, with request bodies capped at 256 KB.
Install
Run in a terminal
claude mcp add --transport http mcp-server https://mcp.webcrawlerapi.com/mcpAdd to Settings → Connectors → Add custom connector
https://mcp.webcrawlerapi.com/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"mcp-server": {
"url": "https://mcp.webcrawlerapi.com/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"mcp-server": {
"type": "http",
"url": "https://mcp.webcrawlerapi.com/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"mcp-server": {
"serverUrl": "https://mcp.webcrawlerapi.com/mcp"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"mcp-server": {
"type": "http",
"url": "https://mcp.webcrawlerapi.com/mcp"
}
}
}Tools this server provides (1)
Taken from the project documentation; the running server may differ.
- get_markdown
- Fetches a public web page and returns its main content as Markdown.
What it does
- Page to Markdown
- Reads any public http or https page and returns its main content.
- JavaScript rendering
- Pages that render client-side are supported.
- OAuth sign-in
- Authenticate with your account; no API key to paste.
- Clear error messages
- Failures such as blocked pages or empty balance return short explanations.
- Read-only tool
- The tool is marked read-only and idempotent.
- Many clients
- Setup steps for Claude, Cursor, VS Code, ChatGPT and MCP Inspector.
Good for
- Pull documentation pages into a chat for summarising
- Gather article text for a knowledge base
- Read product pages without ads and navigation noise
- Let an assistant check a live page before answering
Pricing
Paid, from $99 a month
Pay per page from $0.002; Professional $99/mo, Business $499/mo
Prices change. Check webcrawlerapi.com before you buy.
WebCrawlerAPI FAQ
Do I need an API key?
No. Sign-in is OAuth; you approve access for one of your organizations.
What does a call cost?
Calls are billed to your organization. The pricing page lists pay as you go from $0.002 a page and $99 a month for Professional.
What are the limits?
120 requests per minute per organization, 300 per minute per IP, and 256 KB request bodies.
Which pages can it read?
Public http or https pages. Sites that block the request or robots.txt return an error.
Is WebCrawlerAPI free?
No. WebCrawlerAPI is a paid product. Paid plans start at $99 a month.
How do I add WebCrawlerAPI to Claude Code?
Run this command in your terminal: claude mcp add --transport http mcp-server https://mcp.webcrawlerapi.com/mcp
What are the best alternatives to WebCrawlerAPI?
The closest alternatives to WebCrawlerAPI are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.
Alternatives to WebCrawlerAPI

Scrapling MCP Server
MCP serverMCP server for AI clients to scrape sites with HTTP and stealth browser sessions
Runs locally- 17 tools
- Open source
- 104K GitHub stars
- Web search and scraping

Firecrawl MCP Server
MCP serverMCP server that lets AI agents search, scrape and interact with web pages
- 3 tools
- Remote
- Freemium
- 7.8K GitHub stars
- Web search and scraping

SearXNG Search
MCP serverMCP server for web search and page reading through a SearXNG instance
Runs locally- 3 tools
- Open source
- 1.4K GitHub stars
- Web search and scraping

NeuralVerge
MCP serverStructured, source-tagged business and people data from 300+ sources for agents
- 200 tools
- Remote
- Free trial
- 1 GitHub star
- Web search and scraping

AI-Powered directory
MCP serverSearch AI tools, agents and MCP servers and get install configs from inside your AI assistant.
- 4 tools
- Remote
- Free
- Developer tools

Shopify MCP Server (any store)
MCP serverHosted MCP server that returns any Shopify store's products, prices and stock
Not checked yet- 4 tools
- Remote
- Paid
- E-commerce