Skip to content

MCP servers for web search and scraping

199 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  3. Search AI tools, agents and MCP servers and get install configs from inside your AI assistant.

    • 4 tools
    • Remote
    • Free
    • Developer tools
  4. Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  5. Search a catalogue of Apify Actors and n8n recipes for web-data jobs

    • 8 tools
    • Remote
    • Free
    • Web search and scraping
  6. Look up the official API and ranked scraping tools for 172 websites

    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  7. Read public X Articles, video transcripts and posts as plain text, no account needed

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  8. Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  9. Search a free directory of MCP servers and x402 services, or list your own

    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  10. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  11. Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  12. Free check of whether GPTBot, ClaudeBot and PerplexityBot can really reach a website

    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  13. Test whether GPTBot, ClaudeBot, PerplexityBot and 10 more crawlers can reach a page

    • 1 tool
    • Remote
    • Free
    • Marketing and SEO
  14. Check whether a half-remembered thing can be found before your agent answers

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  15. Hourly-checked free HTTP and SOCKS proxy lists with a CLI and MCP tools to fetch pages

    Runs locally
    • Free
    • Web search and scraping
  16. Fetch web pages as markdown with SSRF protection, reader mode, feeds and robots checks

    Runs locally
    • 15 tools
    • Open source
    • Web search and scraping
  17. Turn any web page into clean Markdown, design tokens and a link list for LLMs

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Files and documents
  18. Check that links load and quotes really appear on a page, from your assistant

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Developer tools
  19. Self-hosted Google Search Console MCP for analytics, URL inspection and sitemaps

    Not checked yet
    • Remote
    • Open source
    • Marketing and SEO
  20. Search podcasts, videos, Chinese forums and paper citations from your coding agent

    Not checked yet
    • Remote
    • Free
    • Web search and scraping
  21. Trace what really runs in Java, Python, TypeScript and JavaScript, with edge confidence

    Runs locally
    • 13 tools
    • Open source
    • Web search and scraping
  22. Find page-extraction services from a frozen catalogue; a time-limited free experiment

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Web search and scraping
  23. Free lookup an agent can query to see if a site permits AI agents to browse, shop or book

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  24. Search immunology and virology literature, trials and grants across six databases at once

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Health and science
  25. Europe PMC literature search, full text and export for AI agents (Python, MIT)

    Runs locally
    • Open source
    • Health and science
  26. Watch a web page, CSS selector, JSON value or price and get changes by poll or webhook

    Not checked yet
    • 6 tools
    • Remote
    • Free
    • Web search and scraping
  27. Read-only Google Play data for agents: apps, charts, reviews, permissions and data safety

    Runs locally
    • 8 tools
    • Open source
    • E-commerce
  28. Crawl documentation sites into local Markdown that agents search offline

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping
  29. Search expiring and recently dropped domain names by drop date, rank and linking domains

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Marketing and SEO

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.