Skip to content

MCP servers for web search and scraping

37 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  3. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  4. Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  5. Fetch web pages as markdown with SSRF protection, reader mode, feeds and robots checks

    Runs locally
    • 15 tools
    • Open source
    • Web search and scraping
  6. Self-hosted Google Search Console MCP for analytics, URL inspection and sitemaps

    Not checked yet
    • Remote
    • Open source
    • Marketing and SEO
  7. Trace what really runs in Java, Python, TypeScript and JavaScript, with edge confidence

    Runs locally
    • 13 tools
    • Open source
    • Web search and scraping
  8. Europe PMC literature search, full text and export for AI agents (Python, MIT)

    Runs locally
    • Open source
    • Health and science
  9. Read-only Google Play data for agents: apps, charts, reviews, permissions and data safety

    Runs locally
    • 8 tools
    • Open source
    • E-commerce
  10. Crawl documentation sites into local Markdown that agents search offline

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping
  11. Browse login-protected and JavaScript-heavy pages through your own Chrome, Edge or Brave

    Runs locally
    • 9 tools
    • Open source
    • Browser automation
  12. Turns web pages and PDFs into cached Markdown outlines read one section at a time

    Runs locally
    • Open source
    • Web search and scraping
  13. Passive-by-default OSINT lookups for AI agents: DNS, WHOIS, IP, breaches, Shodan and more

    Runs locally
    • 16 tools
    • Open source
    • Security
  14. Local, read-only Google Search Console tools for traffic, indexing and sitemap checks

    Runs locally
    • 6 tools
    • Open source
    • Web search and scraping
  15. Web research for agents that returns cited claims with confidence scores

    Runs locally
    • Open source
    • Web search and scraping
  16. Turn any web page into clean Markdown with one tool, run locally with no API key

    Runs locally
    • 1 tool
    • Open source
    • Files and documents
  17. Docker-run MCP server for finding public emails and subdomains of a domain

    Runs locally
    • 4 tools
    • Open source
    • Security
  18. Docker-run MCP server that identifies the CMS, server and frameworks behind a website

    Runs locally
    • 3 tools
    • Open source
    • Security
  19. Keyless web search, page fetching and document reading that runs locally

    Runs locally
    • 11 tools
    • Open source
    • Web search and scraping
  20. Web-fetch tool that returns block pages as failures, via your own Proxlane gateway

    Runs locally
    • 1 tool
    • Open source
    • Web search and scraping
  21. Open-source MCP server for web search via SearXNG or You.com and Markdown crawling

    Runs locally
    • 3 tools
    • Open source
    • Web search and scraping
  22. Open-source crawler that turns websites into Markdown and JSON, with a local MCP server

    Runs locally
    • Open source
    • Web search and scraping
  23. Checks live whether short .com, .ai, .io and other domains are free; suggests free ones

    Runs locally
    • Open source
    • Developer tools
  24. Look up people, places and organizations in Google's Knowledge Graph from Claude

    Runs locally
    • 2 tools
    • Open source
    • Knowledge and memory
  25. 18 Apify-backed web data tools: WHOIS, DNS, SEO audits, app store ranks, company lookups

    Runs locally
    • 14 tools
    • Open source
    • Web search and scraping
  26. Check whether Chinese, Japanese and Korean page titles fit Google by pixel width

    Runs locally
    • 2 tools
    • Open source
    • Web search and scraping
  27. One AI interface to Search Console, Bing, GA4 and AdSense data, with multi-account support

    Runs locally
    • 5 tools
    • Open source
    • Web search and scraping
  28. Shows which page text AI crawlers miss because it only appears after JavaScript runs

    Runs locally
    • 2 tools
    • Open source
    • Web search and scraping
  29. Bulk domain availability checker for AI assistants, with prices at nine registrars

    Runs locally
    • 1 tool
    • Open source
    • Web search and scraping
  30. Self-hosted SEO crawler with AI-crawler checks that assistants can drive over MCP

    Runs locally
    • Open source
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.