Webcrawl Anything
Search, read, crawl and extract from the web locally, with no API key and sources cited
- Pricing
- Open source
- Platforms
- CLI, macOS, Windows
- Runs
- Locally or Remotely
- Version
- 1.4.8
- Listing updated
- today
Similar MCP servers
About Webcrawl Anything
Webcrawl Anything is an open-source MCP server for web search and scraping. It can run locally on your machine or as a hosted remote server.
By default the engine runs on your own machine through npx @webcrawl/mcp, with no account or key. If you set WEBCRAWL_API_KEY, the same tools run on the hosted webcrawl.app service instead, and WEBCRAWL_MODE chooses local, cloud or auto. The package exits with an error rather than silently calling the cloud if the local engine fails to load. A compact schema profile trims tool descriptions for clients with small context windows.
Beyond plain search it covers structured verticals such as shopping, flights, hotels, real estate, business, security, academic, tech and news. Pages are returned as clean Markdown, including JavaScript-heavy sites, and the parse tools read PDFs with OCR, Office files, YouTube videos and audio or video. Crawls are capped at 25 pages per call, and page monitoring with email or webhook alerts is cloud-only.
An npx webcrawl setup command registers the server and installs an agent skill for Claude Code, Cursor, Codex, Windsurf or Gemini CLI. MIT licensed.
Install
Run in a terminal
claude mcp add --transport http webcrawl https://webcrawl.app/mcpAdd to Settings → Connectors → Add custom connector
https://webcrawl.app/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"webcrawl": {
"url": "https://webcrawl.app/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"webcrawl": {
"type": "http",
"url": "https://webcrawl.app/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"webcrawl": {
"serverUrl": "https://webcrawl.app/mcp"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"webcrawl": {
"type": "http",
"url": "https://webcrawl.app/mcp"
}
}
}Using several servers? Build one config file with the MCP config generator.
Configuration
WEBCRAWL_API_KEYOptional. Leave unset to run fully local. Set a webcrawl.app key to use the hosted cloud.WEBCRAWL_MODEOptional: local, cloud or auto. Default auto runs local unless an API key is set.WEBCRAWL_MCP_SCHEMA_PROFILESet to compact for smaller tool schemas in context-limited clients.
Tools this server provides (10)
Taken from the project documentation; the running server may differ.
- web_search
- Search the web or structured verticals.
- web_read
- Fetch a URL and return clean Markdown.
- web_parse
- Parse PDFs, Office files, YouTube and media into Markdown.
- web_parse_status
- Check the status of a parse job.
- web_map
- Discover all URLs on a domain.
- web_crawl
- Crawl up to 25 pages from a URL.
- web_extract
- Extract structured JSON from a URL.
- web_rerank
- Rerank documents by relevance to a query.
- web_monitor
- Watch pages on a schedule with alerts (cloud only).
- search_categories
- List supported search categories and filters.
What it does
- Local by default
- Runs on your machine with no API key or account.
- Search verticals
- Searches shopping, flights, hotels, real estate, business, security and more.
- Page reading
- Returns clean Markdown from JavaScript-heavy pages.
- Document parsing
- Handles PDFs with OCR, Office files, YouTube and audio or video.
- Structured extraction
- Extracts JSON from a page using a prompt or schema.
- Optional cloud
- Hosted mode adds scheduled monitoring with alerts.
Good for
- Research a topic with cited sources from inside an editor
- Map every URL on a domain before a crawl
- Extract product data from a page into JSON
- Convert a PDF or Word file to Markdown for an agent
Pricing
Open source
MIT licensed; optional hosted cloud with an API key
Webcrawl Anything FAQ
Do I need an API key?
No. Local mode is the default and needs no key or account.
What is cloud-only?
Page monitoring with trigger conditions and email or webhook alerts.
How many pages can it crawl?
Up to 25 pages starting from a URL per crawl call.
What licence is it?
MIT.
Is Webcrawl Anything free?
Yes. Webcrawl Anything is open source, so you can use and self-host it for free.
How do I add Webcrawl Anything to Claude Code?
Run this command in your terminal: claude mcp add --transport http webcrawl https://webcrawl.app/mcp
What are the best alternatives to Webcrawl Anything?
The closest alternatives to Webcrawl Anything are Scrapling MCP Server, Firecrawl MCP Server, Douyin_TikTok_Download_API, SearXNG Search and machine_internet.
Where this comes from
- Description: written from the project's README.
- Install steps: from the official MCP Registry entry.



