# CrawlWitness

> CrawlWitness is a paid MCP server for legal and government. Pay-per-call check of what a domain permits AI crawlers to do, with signed receipts. It runs as a hosted remote server, so there is nothing to install locally.

- **Kind:** MCP server
- **Pricing:** Paid
- **Platforms:** API
- **Website:** https://api.crawlwitness.com
- **MCP Registry name:** com.crawlwitness/oracle
- **Categories:** legal and government, web search and scraping
- **Listing updated:** 2026-11-15

## About

Before training on, retrieving from or indexing a site, a crawler can ask CrawlWitness whether that is allowed. The answer is a yes or no for a given domain, AI bot and intended use (train, RAG or search), derived from robots.txt, IETF AIPREF, W3C TDMRep, noai signals and RSL licences.

What sets it apart is proof. Each verdict can come with a signed receipt and an attestation anchored on Base, so you can later show what a site declared on a given date. The site also sells evidence files aimed at disputes and negotiations.

The MCP tools mirror the REST endpoints and are billed in USDC through the x402 protocol on Base or Solana, with no account needed for pay-per-call use. Monthly plans can also be paid in USDC or by card, and a seven-day trial key is offered. Integrations are listed for Scrapy and Crawlee with Playwright.

## Features

- **Yes/no verdict:** Answers whether a given bot may use a domain for a chosen purpose.
- **Full consent report:** Returns the detailed signals behind a verdict.
- **Wallet sanctions check:** Screens a wallet address against OFAC lists.
- **Batch and manifest calls:** Checks up to 100 domains or wallets, or up to 1000 URLs, per request.
- **Signed receipts:** Verdicts can be anchored on Base for later proof.
- **x402 payments:** Agents pay per call in USDC with no sign-up.

## Good for

- Gate a Scrapy or Crawlee crawler on consent signals
- Record what a site declared before using its pages in a RAG index
- Screen a counterparty wallet before a payment
- Keep evidence for a licensing dispute

## Install

### Claude Code (Terminal)

```sh
claude mcp add --transport http oracle https://api.crawlwitness.com/mcp
```

### Claude Desktop (Settings → Connectors → Add custom connector)

```sh
https://api.crawlwitness.com/mcp
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "oracle": {
      "url": "https://api.crawlwitness.com/mcp"
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "oracle": {
      "type": "http",
      "url": "https://api.crawlwitness.com/mcp"
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "oracle": {
      "serverUrl": "https://api.crawlwitness.com/mcp"
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "oracle": {
      "type": "http",
      "url": "https://api.crawlwitness.com/mcp"
    }
  }
}
```

## Tools

- `can_ai_use`: Returns a yes/no verdict for a domain, AI bot and use.
- `check_ai_consent`: Returns the full AI consent report for a domain.
- `check_wallet`: Checks a wallet against OFAC sanctions.

## FAQ

### How much does a verdict cost?

The pricing page lists $0.001 for a yes/no verdict and $0.005 for a full consent report.

### Is there a free trial?

Yes, a 7-day trial key with no wallet or card.

### Which signals does it read?

robots.txt, IETF AIPREF, W3C TDMRep, noai and RSL licences.

### Which MCP tools exist?

can_ai_use, check_ai_consent and check_wallet are named.

### Is CrawlWitness free?

No. CrawlWitness is a paid product.

### How do I add CrawlWitness to Claude Code?

Run this command in your terminal: claude mcp add --transport http oracle https://api.crawlwitness.com/mcp

### What are the best alternatives to CrawlWitness?

The closest alternatives to CrawlWitness are Scrapling MCP Server, Firecrawl MCP Server, SearXNG Search, NeuralVerge and AI-Powered directory.

## Alternatives

- [Scrapling MCP Server](https://ai-powered.com/mcp/scrapling-mcp-server): Scrapling MCP Server is an open-source MCP server for web search and scraping.
- [Firecrawl MCP Server](https://ai-powered.com/mcp/firecrawl-mcp-server): Firecrawl MCP Server is a freemium MCP server for web search and scraping.
- [SearXNG Search](https://ai-powered.com/mcp/searxng-search): SearXNG Search is an open-source MCP server for web search and scraping.
- [NeuralVerge](https://ai-powered.com/mcp/neuralverge): NeuralVerge is an MCP server for web search and scraping.
- [AI-Powered directory](https://ai-powered.com/mcp/ai-powered-directory): AI-Powered directory is a free MCP server for developer tools.
- [NY LLC Publication (FileMyPublication)](https://ai-powered.com/mcp/ny-llc-publication-filemypublication): NY LLC Publication (FileMyPublication) is a free MCP server for legal and government.
- [TribuJus](https://ai-powered.com/mcp/tribujus): TribuJus is an open-source MCP server for legal and government.
- [SupplyClub Catalog](https://ai-powered.com/mcp/supplyclub-catalog): SupplyClub Catalog is a free MCP server for legal and government.

---
Source: https://ai-powered.com/mcp/crawlwitness
