searxNcrawl
Open-source MCP server for web search via SearXNG or You.com and Markdown crawling
- Pricing
- Open source
- Platforms
- CLI, self-hosted, API
- Runs
- Locally
- Version
- 0.33.1
- Listing updated
- 2 days ago
About searxNcrawl
searxNcrawl is an open-source MCP server for web search and scraping. It runs locally on your machine and is started by your AI client.
searxNcrawl pairs search with crawling so an agent can go from a query to clean page text in one server. Search queries your own SearXNG instance and, if you add a key, the You.com API in parallel, then merges, deduplicates and attributes results to their provider, with language and time-range filters. With include_content set, each result also carries its page as Markdown.
Crawling is built on Crawl4AI and Playwright. It converts one or many URLs to Markdown or JSON, strips repeated blocks, can remove links, and can crawl a whole site with depth and page limits aimed at documentation. For pages behind a login, crawl-capture records a session once so later crawls can reuse it. Chromium downloads on the first crawl, about 550 MB.
Run uvx searxncrawl setup to register it with Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop; it backs up each file before changing it. It runs over STDIO or HTTP, needs no API key when you use your own SearXNG, and is MIT licensed. The site mentions a managed Cloud version as planned, with a waitlist, and describes it as a Firecrawl-compatible replacement.
Install
Run in a terminal
claude mcp add searxncrawl --env SEARXNG_PASSWORD='<SEARXNG_PASSWORD>' --env SEARXNG_EXTRA_HEADERS='<SEARXNG_EXTRA_HEADERS>' --env YDC_API_KEY='<YDC_API_KEY>' -- uvx searxncrawlAdd to claude_desktop_config.json
{
"mcpServers": {
"searxncrawl": {
"command": "uvx",
"args": [
"searxncrawl"
],
"env": {
"SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
"SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
"YDC_API_KEY": "<YDC_API_KEY>"
}
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"searxncrawl": {
"command": "uvx",
"args": [
"searxncrawl"
],
"env": {
"SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
"SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
"YDC_API_KEY": "<YDC_API_KEY>"
}
}
}
}Add to .vscode/mcp.json
{
"servers": {
"searxncrawl": {
"type": "stdio",
"command": "uvx",
"args": [
"searxncrawl"
],
"env": {
"SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
"SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
"YDC_API_KEY": "<YDC_API_KEY>"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"searxncrawl": {
"command": "uvx",
"args": [
"searxncrawl"
],
"env": {
"SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
"SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
"YDC_API_KEY": "<YDC_API_KEY>"
}
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"searxncrawl": {
"command": "uvx",
"args": [
"searxncrawl"
],
"env": {
"SEARXNG_PASSWORD": "<SEARXNG_PASSWORD>",
"SEARXNG_EXTRA_HEADERS": "<SEARXNG_EXTRA_HEADERS>",
"YDC_API_KEY": "<YDC_API_KEY>"
}
}
}
}Configuration
SEARXNG_URLURL of your SearXNG instance with JSON output enabled; enables SearXNG search (no default)SEARXNG_USERNAMEOptional basic auth user for SearXNG (set both or neither)SEARXNG_PASSWORDOptional basic auth password for SearXNG (set both or neither)SEARXNG_EXTRA_HEADERSJSON object of SearXNG-only ASCII headers; values may reference @/absolute or @~/ secret files; invalid values disable SearXNGYDC_API_KEYYou.com API key; enables You.com search (queries go to You.com, US)SEARCH_RESULT_FIELDSComma-separated result fields: title,url,content,publishedDate,providers (unset = all)
Tools this server provides (3)
Taken from the project documentation; the running server may differ.
- search
- Query the configured search providers and return merged results.
- crawl
- Crawl one or more URLs to Markdown or JSON.
- crawl_site
- Crawl a site into one Markdown document per page.
What it does
- Merged search
- SearXNG and optional You.com results combined and attributed.
- Page crawl
- One or many URLs to clean Markdown or JSON.
- Site crawl
- Whole-site crawls with depth and page limits.
- Authenticated crawls
- Capture a login session once and crawl behind it.
- One-step setup
- uvx searxncrawl setup configures supported agents with backups.
- STDIO and HTTP
- Run locally or expose an HTTP endpoint with host and CORS guards.
- CLI and Python API
- Use the same engine from a shell or from code.
Good for
- Give a coding agent web search without a paid search API
- Turn a documentation site into Markdown files
- Crawl pages that require a login
- Feed an LLM clean text for research
- Self-host search and scraping for a team
Pricing
Open source
MIT licence; managed Cloud planned, price not listed
Prices change. Check searxncrawl.io before you buy.
searxNcrawl FAQ
Do I need an API key?
Not with your own SearXNG instance. A You.com key is optional and enables You.com search.
Does crawling work without a search provider?
Yes. Crawling works without either provider.
How big is the first install?
Chromium for Playwright is downloaded on the first crawl, about 550 MB.
Is there a hosted version?
A managed Cloud version is planned and has a waitlist.
Is searxNcrawl free?
Yes. searxNcrawl is open source, so you can use and self-host it for free.
How do I add searxNcrawl to Claude Code?
Run this command in your terminal: claude mcp add searxncrawl --env SEARXNG_PASSWORD='<SEARXNG_PASSWORD>' --env SEARXNG_EXTRA_HEADERS='<SEARXNG_EXTRA_HEADERS>' --env YDC_API_KEY='<YDC_API_KEY>' -- uvx searxncrawl
What are the best alternatives to searxNcrawl?
The closest alternatives to searxNcrawl are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.
Alternatives to searxNcrawl

Scrapling MCP Server
MCP serverMCP server for AI clients to scrape sites with HTTP and stealth browser sessions
Runs locally- 17 tools
- Open source
- 104K GitHub stars
- Web search and scraping

Firecrawl MCP Server
MCP serverMCP server that lets AI agents search, scrape and interact with web pages
- 3 tools
- Remote
- Freemium
- 7.8K GitHub stars
- Web search and scraping

SearXNG Search
MCP serverMCP server for web search and page reading through a SearXNG instance
Runs locally- 3 tools
- Open source
- 1.4K GitHub stars
- Web search and scraping

NeuralVerge
MCP serverStructured, source-tagged business and people data from 300+ sources for agents
- 200 tools
- Remote
- Free trial
- 1 GitHub star
- Web search and scraping

AI-Powered directory
MCP serverSearch AI tools, agents and MCP servers and get install configs from inside your AI assistant.
- 4 tools
- Remote
- Free
- Developer tools

Shopify MCP Server (any store)
MCP serverHosted MCP server that returns any Shopify store's products, prices and stock
Not checked yet- 4 tools
- Remote
- Paid
- E-commerce