Skip to content

MCP servers for web search and scraping

9 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  2. Hourly-checked free HTTP and SOCKS proxy lists with a CLI and MCP tools to fetch pages

    Runs locally
    • Free
    • Web search and scraping
  3. Search podcasts, videos, Chinese forums and paper citations from your coding agent

    Not checked yet
    • Remote
    • Free
    • Web search and scraping
  4. Live Google results and domain rank checks by country and device, pay per search

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  5. Scrape, crawl, search and extract web pages as clean Markdown or JSON for agents

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  6. Check which AI crawlers can read a site and generate llms.txt, robots.txt and schema

    Runs locally
    • 9 tools
    • Freemium
    • Web search and scraping
  7. Ask Claude about Reddit and Hacker News threads you exported with Comment Scraper

    Runs locally
    • 4 tools
    • Freemium
    • Web search and scraping
  8. Local research workspace that tracks sources, claims and gaps for your AI agent

    Runs locally
    • 5 tools
    • Free
    • Web search and scraping
  9. Reads web pages with the cheapest tool that works, then checks the page is real

    Runs locally
    • Free
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.