# Atlas Web Compiler & Scraper

> Atlas Web Compiler & Scraper is a freemium MCP server for web search and scraping. Zero-LLM web compiler MCP that turns pages into clean Markdown and typed structure. It can run locally on your machine or as a hosted remote server.

- **Kind:** MCP server
- **Pricing:** Freemium
- **Platforms:** CLI, API
- **Website:** https://atlas-compiler.com
- **Repository:** https://github.com/atlas-compiler/atlas/tree/HEAD/mcp
- **MCP Registry name:** com.atlas-compiler/atlas
- **Categories:** web search and scraping
- **Listing updated:** 2026-11-14

## About

Where most scrapers hand back page-shaped noise, or ask a model to guess structure on every call, Atlas runs pages through a fixed pipeline of fetch, parse, analyze, optimize and emit. The output is token-efficient Markdown plus a typed intermediate representation: a content graph of nodes with reading order and provenance back to the source tag and selector. Navigation and page chrome are marked as excluded.

Through MCP an assistant can scrape a single page, map a site, crawl within bounds or batch many URLs. The README says long outputs get a table of contents, section splitting and explicit truncation banners to protect the model's context window, and that no LLM is involved in the extraction, so results are reproducible. Repeating an identical request within the cache window returns the cached result for no credit.

It runs as a local stdio server through npm (`@atlascompiler/mcp`, which needs an `ATLAS_API_KEY`) or as a remote Streamable HTTP endpoint. An init wizard configures Claude Desktop, Cursor, VS Code and Windsurf. The README says the CLI refuses API keys passed as flags, and outbound requests go through a separate egress service for SSRF protection.

## Features

- **Deterministic compilation:** Fixed, versioned transformations produce the same Markdown and typed output for the same page.
- **Typed content graph:** Nodes carry reading order, hierarchy and provenance back to the source tag and selector.
- **Scrape, map, crawl, batch:** One engine covers single pages, whole-site discovery, bounded crawls and batches of URLs.
- **Context protection:** Automatic table of contents, section splitting, token budgeting and truncation banners.
- **Setup wizard:** An init command validates the API key and configures supported editors without writing the key into their config files.
- **Cached replays:** Identical requests within the cache TTL return stored results at zero credit cost.

## Good for

- Feed clean documentation pages to an assistant without nav and boilerplate
- Map the URLs of a site before crawling it
- Crawl a domain up to a page limit into Markdown
- Batch-compile a list of URLs for a knowledge base
- Keep structured, source-linked web context in a data pipeline

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http atlas https://api.atlas-compiler.com/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://api.atlas-compiler.com/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "atlas": {
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "atlas": {
      "type": "http",
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "atlas": {
      "serverUrl": "https://api.atlas-compiler.com/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "atlas": {
      "type": "http",
      "url": "https://api.atlas-compiler.com/mcp"
    }
  }
}
```

## FAQ

### Does it use an LLM to extract content?

No. The README describes a zero-LLM pipeline, so extraction is deterministic.

### What does it cost?

A Free plan gives 250 compute credits a month with no card; paid tiers raise credits, rate limits and pages per crawl. One credit covers one compiled page or uncached fetch.

### How do I install it?

Run `npx -y @atlascompiler/mcp init` to set up editors, or use the remote endpoint at api.atlas-compiler.com/mcp.

### What happens when I hit my quota?

Requests get an HTTP 429 with retry-after headers; extra concurrent jobs are queued rather than dropped.

### Is Atlas Web Compiler & Scraper free?

Atlas Web Compiler & Scraper has a free plan, with paid plans for more usage or features.

### How do I add Atlas Web Compiler & Scraper to Claude Code?

Run this command in your terminal: claude mcp add --transport http atlas https://api.atlas-compiler.com/mcp

### What are the best alternatives to Atlas Web Compiler & Scraper?

The closest alternatives to Atlas Web Compiler & Scraper are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [Shopify MCP Server (any store)](https://ai-powered.com/mcp/shopify-mcp-server-any-store): Shopify MCP Server (any store) is a paid MCP server for e-commerce.
- [Chanroll: YouTube Channel Video Scraper (Official API)](https://ai-powered.com/mcp/chanroll-youtube-channel-video-scraper-official-api): Chanroll: YouTube Channel Video Scraper (Official API) is a paid MCP server for design and media.
- [Vidvett: YouTube Video Details Scraper (Official API)](https://ai-powered.com/mcp/vidvett-youtube-video-details-scraper-official-api): Vidvett: YouTube Video Details Scraper (Official API) is a paid MCP server for design and media.

---
Source: https://ai-powered.com/mcp/atlas-web-compiler-and-scraper
