# searxNcrawl

> searxNcrawl is an open-source MCP server for web search and scraping. Open-source MCP server for web search via SearXNG or You.com and Markdown crawling. It runs locally on your machine and is started by your AI client.

- **Kind:** MCP server
- **Pricing:** Open source
- **Platforms:** CLI, self-hosted, API
- **Website:** https://www.searxncrawl.io
- **Repository:** https://github.com/dasdigitalemomentum/searxncrawl
- **MCP Registry name:** io.github.DasDigitaleMomentum/searxncrawl
- **Categories:** web search and scraping
- **Listing updated:** 2026-11-14

## About

searxNcrawl pairs search with crawling so an agent can go from a query to clean page text in one server. Search queries your own SearXNG instance and, if you add a key, the You.com API in parallel, then merges, deduplicates and attributes results to their provider, with language and time-range filters. With include_content set, each result also carries its page as Markdown.

Crawling is built on Crawl4AI and Playwright. It converts one or many URLs to Markdown or JSON, strips repeated blocks, can remove links, and can crawl a whole site with depth and page limits aimed at documentation. For pages behind a login, `crawl-capture` records a session once so later crawls can reuse it. Chromium downloads on the first crawl, about 550 MB.

Run `uvx searxncrawl setup` to register it with Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop; it backs up each file before changing it. It runs over STDIO or HTTP, needs no API key when you use your own SearXNG, and is MIT licensed. The site mentions a managed Cloud version as planned, with a waitlist, and describes it as a Firecrawl-compatible replacement.

## Features

- **Merged search:** SearXNG and optional You.com results combined and attributed.
- **Page crawl:** One or many URLs to clean Markdown or JSON.
- **Site crawl:** Whole-site crawls with depth and page limits.
- **Authenticated crawls:** Capture a login session once and crawl behind it.
- **One-step setup:** uvx searxncrawl setup configures supported agents with backups.
- **STDIO and HTTP:** Run locally or expose an HTTP endpoint with host and CORS guards.
- **CLI and Python API:** Use the same engine from a shell or from code.

## Good for

- Give a coding agent web search without a paid search API
- Turn a documentation site into Markdown files
- Crawl pages that require a login
- Feed an LLM clean text for research
- Self-host search and scraping for a team

## Install

### Claude Code (Terminal)

```sh
claude mcp add searxncrawl --env SEARXNG_PASSWORD='<SEARXNG_PASSWORD>' --env SEARXNG_EXTRA_HEADERS='<SEARXNG_EXTRA_HEADERS>' --env YDC_API_KEY='<YDC_API_KEY>' -- uvx searxncrawl
```

### Claude Desktop (claude_desktop_config.json)

```json
{
  "mcpServers": {
    "searxncrawl": {
      "command": "uvx",
      "args": [
        "searxncrawl"
      ],
      "env": {
        "SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
        "SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
        "YDC_API_KEY": "<YDC_API_KEY>"
      }
    }
  }
}
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "searxncrawl": {
      "command": "uvx",
      "args": [
        "searxncrawl"
      ],
      "env": {
        "SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
        "SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
        "YDC_API_KEY": "<YDC_API_KEY>"
      }
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "searxncrawl": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "searxncrawl"
      ],
      "env": {
        "SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
        "SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
        "YDC_API_KEY": "<YDC_API_KEY>"
      }
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "searxncrawl": {
      "command": "uvx",
      "args": [
        "searxncrawl"
      ],
      "env": {
        "SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
        "SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
        "YDC_API_KEY": "<YDC_API_KEY>"
      }
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "searxncrawl": {
      "command": "uvx",
      "args": [
        "searxncrawl"
      ],
      "env": {
        "SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
        "SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
        "YDC_API_KEY": "<YDC_API_KEY>"
      }
    }
  }
}
```

## Tools

- `search`: Query the configured search providers and return merged results.
- `crawl`: Crawl one or more URLs to Markdown or JSON.
- `crawl_site`: Crawl a site into one Markdown document per page.

## FAQ

### Do I need an API key?

Not with your own SearXNG instance. A You.com key is optional and enables You.com search.

### Does crawling work without a search provider?

Yes. Crawling works without either provider.

### How big is the first install?

Chromium for Playwright is downloaded on the first crawl, about 550 MB.

### Is there a hosted version?

A managed Cloud version is planned and has a waitlist.

### Is searxNcrawl free?

Yes. searxNcrawl is open source, so you can use and self-host it for free.

### How do I add searxNcrawl to Claude Code?

Run this command in your terminal: claude mcp add searxncrawl --env SEARXNG_PASSWORD='<SEARXNG_PASSWORD>' --env SEARXNG_EXTRA_HEADERS='<SEARXNG_EXTRA_HEADERS>' --env YDC_API_KEY='<YDC_API_KEY>' -- uvx searxncrawl

### What are the best alternatives to searxNcrawl?

The closest alternatives to searxNcrawl are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [Shopify MCP Server (any store)](https://ai-powered.com/mcp/shopify-mcp-server-any-store): Shopify MCP Server (any store) is a paid MCP server for e-commerce.
- [Chanroll: YouTube Channel Video Scraper (Official API)](https://ai-powered.com/mcp/chanroll-youtube-channel-video-scraper-official-api): Chanroll: YouTube Channel Video Scraper (Official API) is a paid MCP server for design and media.
- [Vidvett: YouTube Video Details Scraper (Official API)](https://ai-powered.com/mcp/vidvett-youtube-video-details-scraper-official-api): Vidvett: YouTube Video Details Scraper (Official API) is a paid MCP server for design and media.

---
Source: https://ai-powered.com/mcp/searxncrawl
