Skip to content

MCP servers for web search and scraping

240 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. Watch web pages from Claude or ChatGPT and read summaries of what changed

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  2. Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  3. Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  4. Read public X Articles, video transcripts and posts as plain text, no account needed

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  5. Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  6. Turn YouTube captions, PDFs and web pages into clean Markdown for RAG

    • Remote
    • Freemium
    • Knowledge and memory
  7. Read Bluesky, Telegram, note.com and WeChat posts without logging in

    • Remote
    • Freemium
    • Web search and scraping
  8. Check which e-commerce stores publish a UCP profile and what it declares

    • Remote
    • Freemium
    • E-commerce
  9. YouTube transcripts, comments, video stats and channel monitoring for AI agents

    • 6 tools
    • Remote
    • Freemium
    • Design and media
  10. Search a free directory of MCP servers and x402 services, or list your own

    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  11. Paid web, OpenAPI and Base chain evidence for agents, billed per call in USDC

    • 12 tools
    • Remote
    • Freemium
    • Web search and scraping
  12. Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  13. Free check of whether GPTBot, ClaudeBot and PerplexityBot can really reach a website

    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  14. Search what was said on YouTube and podcasts and get quotes cued to the second

    • 4 tools
    • Remote
    • Freemium
    • Design and media
  15. Web search, page extraction and structured live-web datasets for agents

    • 3 tools
    • Remote
    • Freemium
    • Web search and scraping
  16. Watch chosen fields on web pages and have an assistant create monitors and read changes

    • 9 tools
    • Remote
    • Freemium
    • Web search and scraping
  17. Website screenshots for AI agents: full page, ad and cookie-banner removal

    • Remote
    • Freemium
    • Web search and scraping
  18. Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  19. Research African companies from an assistant, with cited answers and spend approval

    Not checked yet
    • 2 tools
    • Remote
    • Freemium
    • Sales and CRM
  20. Test whether GPTBot, ClaudeBot, PerplexityBot and 10 more crawlers can reach a page

    • 1 tool
    • Remote
    • Free
    • Marketing and SEO
  21. Crawl and scrape sites to markdown, then ask what changed between runs

    Not checked yet
    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  22. Turn YouTube, TikTok, Instagram, Facebook, X and article URLs into transcripts and JSON

    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  23. Check whether a half-remembered thing can be found before your agent answers

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  24. Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  25. Web capture, MCP and x402 inspectors and a verified directory for agent tools

    • 34 tools
    • Remote
    • Freemium
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.