# Crawlforge MCP Server

> Crawlforge MCP Server is a freemium MCP server for web search and scraping. 31 web scraping, search and research tools that return clean Markdown or JSON. It runs locally on your machine and is started by your AI client.

- **Kind:** MCP server
- **Pricing:** Freemium
- **Platforms:** API, CLI
- **Website:** https://www.crawlforge.dev
- **Repository:** https://github.com/mysleekdesigns/crawlforge-mcp
- **MCP Registry name:** io.github.mysleekdesigns/crawlforge-mcp-server
- **Categories:** web search and scraping, browser automation
- **Listing updated:** 2026-11-14

## About

CrawlForge prices every tool in credits, which shapes how an agent should use it. A page fetch costs 1 credit, a full scrape 2, a web search 5 and a deep research run 10, so 1,000 free credits cover about 1,000 fetches or 100 research runs. Every call needs a CRAWLFORGE_API_KEY; sign-in through crawlforge login keeps the key out of the terminal.

The tools span basic extraction, site mapping, PDF and document processing, schema-based and LLM extraction, change tracking with hosted monitors, stealth rendering behind bot walls, scripted clicks and form fills, stateful browser sessions, batch scraping and an autonomous agent that works from a prompt with hard limits on steps, URLs and time. Pre-built scrapers cover GitHub, Amazon, LinkedIn and seven more sites, and serp_rank checks Google rank through the optional DataForSEO account.

extract_with_llm uses a local Ollama model by default, so no separate LLM key is needed. The README lists an MIT licence and SSRF protection on requests.

## Features

- **Credit-priced tools:** Costs range from 1 credit for fetches to 10 for deep research.
- **Local LLM extraction:** Runs extraction on a local Ollama model by default.
- **Stealth and actions:** Render behind anti-bot walls and script clicks, scrolls and forms.
- **Change monitoring:** Hosted monitors check pages on a schedule and alert on change.
- **Autonomous agent:** Plans and gathers from a natural-language prompt under hard limits.
- **Site templates:** Pre-built scrapers for GitHub, Amazon, LinkedIn and others.

## Good for

- Turn a URL into clean Markdown for an agent
- Monitor competitor pricing pages for changes
- Crawl a site and map its pages
- Run multi-source research with conflict detection
- Check Google rank for a keyword

## Install

### Claude Code (Terminal)

```sh
claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-server
```

### Claude Desktop (claude_desktop_config.json)

```json
{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "crawlforge-mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}
```

## Tools

- `fetch_url`: Fetch raw or rendered content from a URL.
- `extract_text`: Extract plain text from a page.
- `extract_links`: Extract and categorise links.
- `extract_metadata`: Pull title, description, OG tags and structured data.
- `scrape_template`: Run a pre-built scraper for a known site.
- `list_ollama_models`: List local Ollama models.
- `get_batch_results`: Retrieve results of a batch_scrape job.
- `read_result`: Read back a truncated result by handle.
- `scrape_structured`: Extract structured data from a page.
- `extract_content`: Get clean markdown or text.
- `map_site`: Discover and map pages on a site.
- `process_document`: Process PDFs and other documents.
- `scrape`: Unified multi-format scrape.
- `browser_session`: Keep one browser page alive across calls.
- `track_changes`: Track page changes with baselines and alerts.
- `analyze_content`: Analyse readability, topics and sentiment.
- `extract_structured`: Extract data matching a schema.
- `extract_with_llm`: LLM-powered schema extraction.
- `summarize_content`: Summarise a page.
- `crawl_deep`: Crawl a site following internal links.
- `stealth_mode`: Render behind anti-bot walls.
- `scrape_with_actions`: Scrape pages needing clicks, scrolls or forms.
- `batch_scrape`: Process many URLs in parallel.
- `search_web`: Search the web and return ranked JSON.
- `serp_rank`: Check a domain's Google rank for a keyword.
- `generate_llms_txt`: Generate an llms.txt file.
- `agent`: Autonomous research and extraction from a prompt.
- `deep_research`: Multi-source research with conflict detection.

## FAQ

### Is there a free tier?

Yes, 1,000 one-time credits with no credit card.

### Do I need an LLM key for extraction?

No, extract_with_llm uses a local Ollama model by default.

### What do the plans cost?

Hobby is $190 a year, Professional $990 and Business $3,990, all billed yearly.

### Is serp_rank included?

It needs your own DataForSEO login and is billed to that account.

### Is Crawlforge MCP Server free?

Crawlforge MCP Server has a free plan, with paid plans for more usage or features.

### How do I add Crawlforge MCP Server to Claude Code?

Run this command in your terminal: claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-server

### What are the best alternatives to Crawlforge MCP Server?

The closest alternatives to Crawlforge MCP Server are Browser Use, Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search and Oya Browser.

## Alternatives

- [Browser Use](https://ai-powered.com/mcp/browser-use): Browser Use is a freemium MCP server for browser automation.
- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [Oya Browser](https://ai-powered.com/mcp/oya-browser): Oya Browser is a freemium MCP server for browser automation.
- [Mantis](https://ai-powered.com/mcp/mantis-2): Mantis is an open-source MCP server for browser automation.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.

---
Source: https://ai-powered.com/mcp/crawlforge-mcp-server
