# Siftdog

> Siftdog is an open-source MCP server for web search and scraping. Self-hosted, Tavily-compatible web search, extract, crawl and map tools for agents. It runs locally on your machine and is started by your AI client.

- **Kind:** MCP server
- **Pricing:** Open source
- **Platforms:** self-hosted, CLI, Linux, macOS
- **Website:** https://siftdog.com
- **Repository:** https://github.com/khsarvar/siftdog
- **MCP Registry name:** io.github.khsarvar/siftdog
- **Categories:** web search and scraping
- **Listing updated:** 2026-11-16

## About

Siftdog reproduces the four Tavily endpoints (search, extract, crawl, map) on infrastructure you run, so code written for the official tavily-python SDK or LangChain works by changing only the base URL. Search goes through a SearXNG metasearch instance with BM25 relevance blended into the engines' order, so no search API key or per-search bill is needed.

Extraction produces clean markdown or text from up to 20 pages per request using trafilatura, and crawling follows links with depth, breadth, limit and regex filters. Optional answers can be written by Claude from retrieved results. The MCP server has four read-only tools over stdio or at /mcp on a running server. Private and internal addresses are blocked, including through redirects and DNS rebinding.

It fetches static HTML only, with no JavaScript rendering, and the site compares itself against Tavily and Firecrawl accordingly. A hosted version is described as planned. Python 3.11+, Docker images for amd64 and arm64, MIT licensed.

## Features

- **Tavily-compatible API:** Same endpoints and response shapes, so existing SDK code keeps working.
- **SearXNG search:** Ranked results with no search API key.
- **Page extraction:** Markdown or text from up to 20 pages per request.
- **Crawl and map:** Depth, breadth and regex filters, or just list URLs.
- **SSRF protection:** Blocks private addresses on connect and every redirect.
- **Docker and pip:** Run the full stack or only extract, crawl and map.

## Good for

- Replace a hosted search API with your own server
- Give a coding agent web search without a vendor key
- Crawl a documentation site into markdown
- List all URLs of a site

## Install

### Claude Code (Terminal)

```sh
claude mcp add siftdog -- uvx siftdog
```

### Claude Desktop (claude_desktop_config.json)

```json
{
  "mcpServers": {
    "siftdog": {
      "command": "uvx",
      "args": [
        "siftdog"
      ]
    }
  }
}
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "siftdog": {
      "command": "uvx",
      "args": [
        "siftdog"
      ]
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "siftdog": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "siftdog"
      ]
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "siftdog": {
      "command": "uvx",
      "args": [
        "siftdog"
      ]
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "siftdog": {
      "command": "uvx",
      "args": [
        "siftdog"
      ]
    }
  }
}
```

## Tools

- `siftdog_search`: Search the web through SearXNG.
- `siftdog_extract`: Extract clean content from pages.
- `siftdog_crawl`: Crawl a site by following links.
- `siftdog_map`: List a site's URLs.

## FAQ

### Is it free?

The software is MIT licensed and self-hosted; you run SearXNG yourself.

### Can it render JavaScript?

No, it fetches static HTML only.

### Which tools does MCP expose?

siftdog_search, siftdog_extract, siftdog_crawl and siftdog_map.

### Is Siftdog free?

Yes. Siftdog is open source, so you can use and self-host it for free.

### How do I add Siftdog to Claude Code?

Run this command in your terminal: claude mcp add siftdog -- uvx siftdog

### What are the best alternatives to Siftdog?

The closest alternatives to Siftdog are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [Shopify MCP Server (any store)](https://ai-powered.com/mcp/shopify-mcp-server-any-store): Shopify MCP Server (any store) is a paid MCP server for e-commerce.
- [Chanroll: YouTube Channel Video Scraper (Official API)](https://ai-powered.com/mcp/chanroll-youtube-channel-video-scraper-official-api): Chanroll: YouTube Channel Video Scraper (Official API) is a paid MCP server for design and media.
- [Vidvett: YouTube Video Details Scraper (Official API)](https://ai-powered.com/mcp/vidvett-youtube-video-details-scraper-official-api): Vidvett: YouTube Video Details Scraper (Official API) is a paid MCP server for design and media.

---
Source: https://ai-powered.com/mcp/siftdog
