Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,212 MCP servers, 15 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–23 of 23

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. ClawGrab

    MCP server

    Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  3. Read-only Google Play data for agents: apps, charts, reviews, permissions and data safety

    Runs locally
    • 8 tools
    • Open source
    • E-commerce
  4. doc-scraper

    MCP server

    Crawl documentation sites into local Markdown that agents search offline

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping
  5. FATSEARCH

    MCP server

    Turns web pages and PDFs into cached Markdown outlines read one section at a time

    Runs locally
    • Open source
    • Web search and scraping
  6. OpenOSINT

    MCP server

    Passive-by-default OSINT lookups for AI agents: DNS, WHOIS, IP, breaches, Shodan and more

    Runs locally
    • 16 tools
    • Open source
    • Security
  7. Local, read-only Google Search Console tools for traffic, indexing and sitemap checks

    Runs locally
    • 6 tools
    • Open source
    • Web search and scraping
  8. Serper.live

    MCP server

    Live Google results and domain rank checks by country and device, pay per search

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  9. webclaw

    MCP server

    Scrape, crawl, search and extract web pages as clean Markdown or JSON for agents

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  10. Docker-run MCP server for finding public emails and subdomains of a domain

    Runs locally
    • 4 tools
    • Open source
    • Security
  11. WhatWeb MCP

    MCP server

    Docker-run MCP server that identifies the CMS, server and frameworks behind a website

    Runs locally
    • 3 tools
    • Open source
    • Security
  12. Free Search

    MCP server

    Keyless web search, page fetching and document reading that runs locally

    Runs locally
    • 11 tools
    • Open source
    • Web search and scraping
  13. Crawl Readiness

    MCP server

    Check which AI crawlers can read a site and generate llms.txt, robots.txt and schema

    Runs locally
    • 9 tools
    • Freemium
    • Web search and scraping
  14. Comment Scraper

    MCP server

    Ask Claude about Reddit and Hacker News threads you exported with Comment Scraper

    Runs locally
    • 4 tools
    • Freemium
    • Web search and scraping
  15. Crawlberg

    MCP server

    Open-source crawler that turns websites into Markdown and JSON, with a local MCP server

    Runs locally
    • Open source
    • Web search and scraping
  16. One AI interface to Search Console, Bing, GA4 and AdSense data, with multi-account support

    Runs locally
    • 5 tools
    • Open source
    • Web search and scraping
  17. WebRouter

    MCP server

    Open source Python interface for web search, extraction and browsing across many providers

    Runs locally
    • Open source
    • Web search and scraping
  18. DomainScout

    MCP server

    Bulk domain availability checker for AI assistants, with prices at nine registrars

    Runs locally
    • 1 tool
    • Open source
    • Web search and scraping
  19. Google News

    MCP server

    MCP server that lets AI assistants search Google News by keyword, date, country or topic

    Runs locally
    • 2 tools
    • Open source
    • Web search and scraping
  20. Self-hosted SEO crawler with AI-crawler checks that assistants can drive over MCP

    Runs locally
    • Open source
    • Web search and scraping
  21. KitchenSink4Web

    MCP server

    Budgeted web page reading for AI assistants, read-only until you allow clicking

    Runs locally
    • 1 tool
    • Open source
    • Browser automation
  22. Keyless web, news, image, video and book search for AI agents, via MCP or REST

    Runs locally
    • Open source
    • Web search and scraping
  23. Siftdog

    MCP server

    Self-hosted, Tavily-compatible web search, extract, crawl and map tools for agents

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping