Skip to content
MCP server

Crawlforge MCP Server

31 web scraping, search and research tools that return clean Markdown or JSON

Runs locallyStatus and history
Pricing
Freemium
Platforms
API, CLI
Runs
Locally
Version
6.25.1
Listing updated
3 days ago

About Crawlforge MCP Server

Crawlforge MCP Server is a freemium MCP server for web search and scraping. It runs locally on your machine and is started by your AI client.

CrawlForge prices every tool in credits, which shapes how an agent should use it. A page fetch costs 1 credit, a full scrape 2, a web search 5 and a deep research run 10, so 1,000 free credits cover about 1,000 fetches or 100 research runs. Every call needs a CRAWLFORGE_API_KEY; sign-in through crawlforge login keeps the key out of the terminal.

The tools span basic extraction, site mapping, PDF and document processing, schema-based and LLM extraction, change tracking with hosted monitors, stealth rendering behind bot walls, scripted clicks and form fills, stateful browser sessions, batch scraping and an autonomous agent that works from a prompt with hard limits on steps, URLs and time. Pre-built scrapers cover GitHub, Amazon, LinkedIn and seven more sites, and serp_rank checks Google rank through the optional DataForSEO account.

extract_with_llm uses a local Ollama model by default, so no separate LLM key is needed. The README lists an MIT licence and SSRF protection on requests.

Install

Run in a terminal

claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-server

Add to claude_desktop_config.json

{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "crawlforge-mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "crawlforge-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "crawlforge-mcp-server"
      ],
      "env": {
        "CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
        "DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
        "DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
      }
    }
  }
}

Configuration

  • CRAWLFORGE_API_KEYCrawlForge API key for authentication and usage/credit tracking. Get a free key at https://www.crawlforge.devRequired
  • DATAFORSEO_LOGINDataForSEO account login. Optional — only needed to enable the serp_rank tool (real Google organic rank lookups); billed separately to the caller's own DataForSEO account.
  • DATAFORSEO_PASSWORDDataForSEO account password. Optional — only needed to enable the serp_rank tool.

Tools this server provides (28)

Taken from the project documentation; the running server may differ.

fetch_url
Fetch raw or rendered content from a URL.
extract_text
Extract plain text from a page.
extract_links
Extract and categorise links.
extract_metadata
Pull title, description, OG tags and structured data.
scrape_template
Run a pre-built scraper for a known site.
list_ollama_models
List local Ollama models.
get_batch_results
Retrieve results of a batch_scrape job.
read_result
Read back a truncated result by handle.
scrape_structured
Extract structured data from a page.
extract_content
Get clean markdown or text.
map_site
Discover and map pages on a site.
process_document
Process PDFs and other documents.
scrape
Unified multi-format scrape.
browser_session
Keep one browser page alive across calls.
track_changes
Track page changes with baselines and alerts.
analyze_content
Analyse readability, topics and sentiment.
extract_structured
Extract data matching a schema.
extract_with_llm
LLM-powered schema extraction.
summarize_content
Summarise a page.
crawl_deep
Crawl a site following internal links.
stealth_mode
Render behind anti-bot walls.
scrape_with_actions
Scrape pages needing clicks, scrolls or forms.
batch_scrape
Process many URLs in parallel.
search_web
Search the web and return ranked JSON.
serp_rank
Check a domain's Google rank for a keyword.
generate_llms_txt
Generate an llms.txt file.
agent
Autonomous research and extraction from a prompt.
deep_research
Multi-source research with conflict detection.

What it does

Credit-priced tools
Costs range from 1 credit for fetches to 10 for deep research.
Local LLM extraction
Runs extraction on a local Ollama model by default.
Stealth and actions
Render behind anti-bot walls and script clicks, scrolls and forms.
Change monitoring
Hosted monitors check pages on a schedule and alert on change.
Autonomous agent
Plans and gathers from a natural-language prompt under hard limits.
Site templates
Pre-built scrapers for GitHub, Amazon, LinkedIn and others.

Good for

  • Turn a URL into clean Markdown for an agent
  • Monitor competitor pricing pages for changes
  • Crawl a site and map its pages
  • Run multi-source research with conflict detection
  • Check Google rank for a keyword

Pricing

Freemium

1,000 free credits; yearly plans from $190/year

Prices change. Check crawlforge.dev before you buy.

Crawlforge MCP Server FAQ

Is there a free tier?

Yes, 1,000 one-time credits with no credit card.

Do I need an LLM key for extraction?

No, extract_with_llm uses a local Ollama model by default.

What do the plans cost?

Hobby is $190 a year, Professional $990 and Business $3,990, all billed yearly.

Is serp_rank included?

It needs your own DataForSEO login and is billed to that account.

Is Crawlforge MCP Server free?

Crawlforge MCP Server has a free plan, with paid plans for more usage or features.

How do I add Crawlforge MCP Server to Claude Code?

Run this command in your terminal: claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-server

What are the best alternatives to Crawlforge MCP Server?

The closest alternatives to Crawlforge MCP Server are Browser Use, Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search and Oya Browser.

Alternatives to Crawlforge MCP Server

  1. Browser Use

    MCP server

    Browser agents and cloud browsers for AI, with an MCP server for web tasks

    • 7 tools
    • Remote
    • Freemium
    • 121K GitHub stars
    • Browser automation
  2. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  3. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  4. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  5. Oya Browser

    MCP server

    HIPAA-oriented agentic browser with replayable playbooks, reachable over MCP

    • 32 tools
    • Remote
    • Freemium
    • 349 GitHub stars
    • Browser automation
  6. Mantis

    MCP server

    Browser MCP server that gives agents numbered text pages instead of screenshots

    Runs locally
    • 1 tool
    • Open source
    • 5 Hacker News points
    • Browser automation