Crawlforge MCP Server
31 web scraping, search and research tools that return clean Markdown or JSON
- Pricing
- Freemium
- Platforms
- API, CLI
- Runs
- Locally
- Version
- 6.25.1
- Listing updated
- 3 days ago
About Crawlforge MCP Server
Crawlforge MCP Server is a freemium MCP server for web search and scraping. It runs locally on your machine and is started by your AI client.
CrawlForge prices every tool in credits, which shapes how an agent should use it. A page fetch costs 1 credit, a full scrape 2, a web search 5 and a deep research run 10, so 1,000 free credits cover about 1,000 fetches or 100 research runs. Every call needs a CRAWLFORGE_API_KEY; sign-in through crawlforge login keeps the key out of the terminal.
The tools span basic extraction, site mapping, PDF and document processing, schema-based and LLM extraction, change tracking with hosted monitors, stealth rendering behind bot walls, scripted clicks and form fills, stateful browser sessions, batch scraping and an autonomous agent that works from a prompt with hard limits on steps, URLs and time. Pre-built scrapers cover GitHub, Amazon, LinkedIn and seven more sites, and serp_rank checks Google rank through the optional DataForSEO account.
extract_with_llm uses a local Ollama model by default, so no separate LLM key is needed. The README lists an MIT licence and SSRF protection on requests.
Install
Run in a terminal
claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-serverAdd to claude_desktop_config.json
{
"mcpServers": {
"crawlforge-mcp-server": {
"command": "npx",
"args": [
"-y",
"crawlforge-mcp-server"
],
"env": {
"CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
"DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
"DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
}
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"crawlforge-mcp-server": {
"command": "npx",
"args": [
"-y",
"crawlforge-mcp-server"
],
"env": {
"CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
"DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
"DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
}
}
}
}Add to .vscode/mcp.json
{
"servers": {
"crawlforge-mcp-server": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"crawlforge-mcp-server"
],
"env": {
"CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
"DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
"DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"crawlforge-mcp-server": {
"command": "npx",
"args": [
"-y",
"crawlforge-mcp-server"
],
"env": {
"CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
"DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
"DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
}
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"crawlforge-mcp-server": {
"command": "npx",
"args": [
"-y",
"crawlforge-mcp-server"
],
"env": {
"CRAWLFORGE_API_KEY": "<CRAWLFORGE_API_KEY>",
"DATAFORSEO_LOGIN": "<DATAFORSEO_LOGIN>",
"DATAFORSEO_PASSWORD": "<DATAFORSEO_PASSWORD>"
}
}
}
}Configuration
CRAWLFORGE_API_KEYCrawlForge API key for authentication and usage/credit tracking. Get a free key at https://www.crawlforge.devRequiredDATAFORSEO_LOGINDataForSEO account login. Optional — only needed to enable the serp_rank tool (real Google organic rank lookups); billed separately to the caller's own DataForSEO account.DATAFORSEO_PASSWORDDataForSEO account password. Optional — only needed to enable the serp_rank tool.
Tools this server provides (28)
Taken from the project documentation; the running server may differ.
- fetch_url
- Fetch raw or rendered content from a URL.
- extract_text
- Extract plain text from a page.
- extract_links
- Extract and categorise links.
- extract_metadata
- Pull title, description, OG tags and structured data.
- scrape_template
- Run a pre-built scraper for a known site.
- list_ollama_models
- List local Ollama models.
- get_batch_results
- Retrieve results of a batch_scrape job.
- read_result
- Read back a truncated result by handle.
- scrape_structured
- Extract structured data from a page.
- extract_content
- Get clean markdown or text.
- map_site
- Discover and map pages on a site.
- process_document
- Process PDFs and other documents.
- scrape
- Unified multi-format scrape.
- browser_session
- Keep one browser page alive across calls.
- track_changes
- Track page changes with baselines and alerts.
- analyze_content
- Analyse readability, topics and sentiment.
- extract_structured
- Extract data matching a schema.
- extract_with_llm
- LLM-powered schema extraction.
- summarize_content
- Summarise a page.
- crawl_deep
- Crawl a site following internal links.
- stealth_mode
- Render behind anti-bot walls.
- scrape_with_actions
- Scrape pages needing clicks, scrolls or forms.
- batch_scrape
- Process many URLs in parallel.
- search_web
- Search the web and return ranked JSON.
- serp_rank
- Check a domain's Google rank for a keyword.
- generate_llms_txt
- Generate an llms.txt file.
- agent
- Autonomous research and extraction from a prompt.
- deep_research
- Multi-source research with conflict detection.
What it does
- Credit-priced tools
- Costs range from 1 credit for fetches to 10 for deep research.
- Local LLM extraction
- Runs extraction on a local Ollama model by default.
- Stealth and actions
- Render behind anti-bot walls and script clicks, scrolls and forms.
- Change monitoring
- Hosted monitors check pages on a schedule and alert on change.
- Autonomous agent
- Plans and gathers from a natural-language prompt under hard limits.
- Site templates
- Pre-built scrapers for GitHub, Amazon, LinkedIn and others.
Good for
- Turn a URL into clean Markdown for an agent
- Monitor competitor pricing pages for changes
- Crawl a site and map its pages
- Run multi-source research with conflict detection
- Check Google rank for a keyword
Pricing
Freemium
1,000 free credits; yearly plans from $190/year
Prices change. Check crawlforge.dev before you buy.
Crawlforge MCP Server FAQ
Is there a free tier?
Yes, 1,000 one-time credits with no credit card.
Do I need an LLM key for extraction?
No, extract_with_llm uses a local Ollama model by default.
What do the plans cost?
Hobby is $190 a year, Professional $990 and Business $3,990, all billed yearly.
Is serp_rank included?
It needs your own DataForSEO login and is billed to that account.
Is Crawlforge MCP Server free?
Crawlforge MCP Server has a free plan, with paid plans for more usage or features.
How do I add Crawlforge MCP Server to Claude Code?
Run this command in your terminal: claude mcp add crawlforge-mcp-server --env CRAWLFORGE_API_KEY='<CRAWLFORGE_API_KEY>' --env DATAFORSEO_LOGIN='<DATAFORSEO_LOGIN>' --env DATAFORSEO_PASSWORD='<DATAFORSEO_PASSWORD>' -- npx -y crawlforge-mcp-server
What are the best alternatives to Crawlforge MCP Server?
The closest alternatives to Crawlforge MCP Server are Browser Use, Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search and Oya Browser.
Alternatives to Crawlforge MCP Server

Browser Use
MCP serverBrowser agents and cloud browsers for AI, with an MCP server for web tasks
- 7 tools
- Remote
- Freemium
- 121K GitHub stars
- Browser automation

Scrapling MCP Server
MCP serverMCP server for AI clients to scrape sites with HTTP and stealth browser sessions
Runs locally- 17 tools
- Open source
- 104K GitHub stars
- Web search and scraping

Firecrawl MCP Server
MCP serverMCP server that lets AI agents search, scrape and interact with web pages
- 3 tools
- Remote
- Freemium
- 7.8K GitHub stars
- Web search and scraping

SearXNG Search
MCP serverMCP server for web search and page reading through a SearXNG instance
Runs locally- 3 tools
- Open source
- 1.4K GitHub stars
- Web search and scraping

Oya Browser
MCP serverHIPAA-oriented agentic browser with replayable playbooks, reachable over MCP
- 32 tools
- Remote
- Freemium
- 349 GitHub stars
- Browser automation
Mantis
MCP serverBrowser MCP server that gives agents numbered text pages instead of screenshots
Runs locally- 1 tool
- Open source
- 5 Hacker News points
- Browser automation