Skip to content
MCP server

Atlas Web Compiler & Scraper

Zero-LLM web compiler MCP that turns pages into clean Markdown and typed structure

Install
Pricing
Freemium
Platforms
CLI, API
Runs
Locally or Remotely
Version
1.0.5
Listing updated
3 days ago

About Atlas Web Compiler & Scraper

Atlas Web Compiler & Scraper is a freemium MCP server for web search and scraping. It can run locally on your machine or as a hosted remote server.

Where most scrapers hand back page-shaped noise, or ask a model to guess structure on every call, Atlas runs pages through a fixed pipeline of fetch, parse, analyze, optimize and emit. The output is token-efficient Markdown plus a typed intermediate representation: a content graph of nodes with reading order and provenance back to the source tag and selector. Navigation and page chrome are marked as excluded.

Through MCP an assistant can scrape a single page, map a site, crawl within bounds or batch many URLs. The README says long outputs get a table of contents, section splitting and explicit truncation banners to protect the model's context window, and that no LLM is involved in the extraction, so results are reproducible. Repeating an identical request within the cache window returns the cached result for no credit.

It runs as a local stdio server through npm (@atlascompiler/mcp, which needs an ATLAS_API_KEY) or as a remote Streamable HTTP endpoint. An init wizard configures Claude Desktop, Cursor, VS Code and Windsurf. The README says the CLI refuses API keys passed as flags, and outbound requests go through a separate egress service for SSRF protection.

Install

Run in a terminal

claude mcp add --transport http atlas https://api.atlas-compiler.com/mcp

Add to Settings → Connectors → Add custom connector

https://api.atlas-compiler.com/mcp
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "atlas": {
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "atlas": {
      "type": "http",
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "atlas": {
      "serverUrl": "https://api.atlas-compiler.com/mcp"
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "atlas": {
      "type": "http",
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}

Configuration

  • ATLAS_API_KEYYour Atlas API key. Obtain one at https://dashboard.atlas-compiler.comRequired

What it does

Deterministic compilation
Fixed, versioned transformations produce the same Markdown and typed output for the same page.
Typed content graph
Nodes carry reading order, hierarchy and provenance back to the source tag and selector.
Scrape, map, crawl, batch
One engine covers single pages, whole-site discovery, bounded crawls and batches of URLs.
Context protection
Automatic table of contents, section splitting, token budgeting and truncation banners.
Setup wizard
An init command validates the API key and configures supported editors without writing the key into their config files.
Cached replays
Identical requests within the cache TTL return stored results at zero credit cost.

Good for

  • Feed clean documentation pages to an assistant without nav and boilerplate
  • Map the URLs of a site before crawling it
  • Crawl a domain up to a page limit into Markdown
  • Batch-compile a list of URLs for a knowledge base
  • Keep structured, source-linked web context in a data pipeline

Pricing

Freemium

Free plan with 250 credits a month; paid tiers up to Business and Enterprise (prices not published)

Prices change. Check atlas-compiler.com before you buy.

Atlas Web Compiler & Scraper FAQ

Does it use an LLM to extract content?

No. The README describes a zero-LLM pipeline, so extraction is deterministic.

What does it cost?

A Free plan gives 250 compute credits a month with no card; paid tiers raise credits, rate limits and pages per crawl. One credit covers one compiled page or uncached fetch.

How do I install it?

Run `npx -y @atlascompiler/mcp init` to set up editors, or use the remote endpoint at api.atlas-compiler.com/mcp.

What happens when I hit my quota?

Requests get an HTTP 429 with retry-after headers; extra concurrent jobs are queued rather than dropped.

Is Atlas Web Compiler & Scraper free?

Atlas Web Compiler & Scraper has a free plan, with paid plans for more usage or features.

How do I add Atlas Web Compiler & Scraper to Claude Code?

Run this command in your terminal: claude mcp add --transport http atlas https://api.atlas-compiler.com/mcp

What are the best alternatives to Atlas Web Compiler & Scraper?

The closest alternatives to Atlas Web Compiler & Scraper are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

Alternatives to Atlas Web Compiler & Scraper

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  3. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  4. NeuralVerge

    MCP server

    Structured, source-tagged business and people data from 300+ sources for agents

    • 200 tools
    • Remote
    • Free trial
    • 1 GitHub star
    • Web search and scraping
  5. Search AI tools, agents and MCP servers and get install configs from inside your AI assistant.

    • 4 tools
    • Remote
    • Free
    • Developer tools
  6. Hosted MCP server that returns any Shopify store's products, prices and stock

    Not checked yet
    • 4 tools
    • Remote
    • Paid
    • E-commerce