Skip to content

MCP servers for web search and scraping

25 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  3. Mac app so Claude and Cursor can read Reddit, X and LinkedIn through your Chrome login

    Runs locally
    • 12 tools
    • Freemium
    • Browser automation
  4. Read-only Google Play data for agents: apps, charts, reviews, permissions and data safety

    Runs locally
    • 8 tools
    • Open source
    • E-commerce
  5. Crawl documentation sites into local Markdown that agents search offline

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping
  6. Turns web pages and PDFs into cached Markdown outlines read one section at a time

    Runs locally
    • Open source
    • Web search and scraping
  7. Passive-by-default OSINT lookups for AI agents: DNS, WHOIS, IP, breaches, Shodan and more

    Runs locally
    • 16 tools
    • Open source
    • Security
  8. Local, read-only Google Search Console tools for traffic, indexing and sitemap checks

    Runs locally
    • 6 tools
    • Open source
    • Web search and scraping
  9. Live Google results and domain rank checks by country and device, pay per search

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  10. Scrape, crawl, search and extract web pages as clean Markdown or JSON for agents

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  11. Docker-run MCP server for finding public emails and subdomains of a domain

    Runs locally
    • 4 tools
    • Open source
    • Security
  12. Docker-run MCP server that identifies the CMS, server and frameworks behind a website

    Runs locally
    • 3 tools
    • Open source
    • Security
  13. Keyless web search, page fetching and document reading that runs locally

    Runs locally
    • 11 tools
    • Open source
    • Web search and scraping
  14. Check which AI crawlers can read a site and generate llms.txt, robots.txt and schema

    Runs locally
    • 9 tools
    • Freemium
    • Web search and scraping
  15. Ask Claude about Reddit and Hacker News threads you exported with Comment Scraper

    Runs locally
    • 4 tools
    • Freemium
    • Web search and scraping
  16. Open-source crawler that turns websites into Markdown and JSON, with a local MCP server

    Runs locally
    • Open source
    • Web search and scraping
  17. One AI interface to Search Console, Bing, GA4 and AdSense data, with multi-account support

    Runs locally
    • 5 tools
    • Open source
    • Web search and scraping
  18. Open source Python interface for web search, extraction and browsing across many providers

    Runs locally
    • Open source
    • Web search and scraping
  19. Builds, hosts and updates a small business website from a chat with Claude

    Not checked yet
    • Remote
    • Freemium
    • AI models and LLM tooling
  20. Bulk domain availability checker for AI assistants, with prices at nine registrars

    Runs locally
    • 1 tool
    • Open source
    • Web search and scraping
  21. MCP server that lets AI assistants search Google News by keyword, date, country or topic

    Runs locally
    • 2 tools
    • Open source
    • Web search and scraping
  22. Self-hosted SEO crawler with AI-crawler checks that assistants can drive over MCP

    Runs locally
    • Open source
    • Web search and scraping
  23. Budgeted web page reading for AI assistants, read-only until you allow clicking

    Runs locally
    • 1 tool
    • Open source
    • Browser automation
  24. Keyless web, news, image, video and book search for AI agents, via MCP or REST

    Runs locally
    • Open source
    • Web search and scraping
  25. Self-hosted, Tavily-compatible web search, extract, crawl and map tools for agents

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.