Skip to content

MCP servers for web search and scraping

19 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  2. Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  3. Hourly-checked free HTTP and SOCKS proxy lists with a CLI and MCP tools to fetch pages

    Runs locally
    • Free
    • Web search and scraping
  4. Search podcasts, videos, Chinese forums and paper citations from your coding agent

    Not checked yet
    • Remote
    • Free
    • Web search and scraping
  5. Search 1,500+ indie-made products as alternatives to big-name software from your AI

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  6. Search launched products, alternatives and weekly launch boards on MakerRadar

    Not checked yet
    • 14 tools
    • Remote
    • Free
    • Web search and scraping
  7. Check domain availability and registrar prices and look up public domain records

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  8. Find songs by a remembered lyric or title and read lyrics from lyrics.com

    Runs locally
    • 3 tools
    • Free
    • Events and entertainment
  9. Anonymous, no-key MCP server for SEO and AI crawler checks, robots.txt and llms.txt

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  10. Remote MCP server giving AI assistants free checks on websites, IPs, emails and domains

    Not checked yet
    • 7 tools
    • Remote
    • Free
    • Web search and scraping
  11. Query a 30-question benchmark of You.com, Tavily, Exa and Parallel search APIs

    Not checked yet
    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  12. Keyless web search and page reading for agents using r.jina.ai, Wikipedia, HN and GitHub

    Not checked yet
    • 5 tools
    • Remote
    • Free
    • Web search and scraping
  13. Search a catalogue of MCP servers, agent skills and open-source AI projects

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Community directories
  14. Local research workspace that tracks sources, claims and gaps for your AI agent

    Runs locally
    • 5 tools
    • Free
    • Web search and scraping
  15. Private web search with no API key, plus Moshpit name lookups and a pro directory

    Runs locally
    • 7 tools
    • Free
    • Web search and scraping
  16. Reads web pages with the cheapest tool that works, then checks the page is real

    Runs locally
    • Free
    • Web search and scraping
  17. Checks whether a website is down for everyone or only for you, with HTTP code and timing

    Not checked yet
    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  18. 50+ developer utility APIs as tools: SSL, DNS, WHOIS, email checks, HTML to PDF and more

    Runs locally
    • 14 tools
    • Free
    • Developer tools

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.