Skip to content

MCP servers for web search and scraping

247 MCP servers that give AI live web search, page fetching and structured data extraction. Each one shows how to install it in Claude, Cursor and VS Code, whether it runs locally or remotely, and the tools it provides.
  1. Watch web pages from Claude or ChatGPT and read summaries of what changed

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  2. Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  3. Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  4. Read public X Articles, video transcripts and posts as plain text, no account needed

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  5. Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  6. Turn YouTube captions, PDFs and web pages into clean Markdown for RAG

    • Remote
    • Freemium
    • Knowledge and memory
  7. Read Bluesky, Telegram, note.com and WeChat posts without logging in

    • Remote
    • Freemium
    • Web search and scraping
  8. Check which e-commerce stores publish a UCP profile and what it declares

    • Remote
    • Freemium
    • E-commerce
  9. YouTube transcripts, comments, video stats and channel monitoring for AI agents

    • 6 tools
    • Remote
    • Freemium
    • Design and media
  10. Search a free directory of MCP servers and x402 services, or list your own

    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  11. Paid web, OpenAPI and Base chain evidence for agents, billed per call in USDC

    • 12 tools
    • Remote
    • Freemium
    • Web search and scraping
  12. Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  13. Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  14. Free check of whether GPTBot, ClaudeBot and PerplexityBot can really reach a website

    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  15. Search what was said on YouTube and podcasts and get quotes cued to the second

    • 4 tools
    • Remote
    • Freemium
    • Design and media
  16. Web search, page extraction and structured live-web datasets for agents

    • 3 tools
    • Remote
    • Freemium
    • Web search and scraping
  17. Watch chosen fields on web pages and have an assistant create monitors and read changes

    • 9 tools
    • Remote
    • Freemium
    • Web search and scraping
  18. Website screenshots for AI agents: full page, ad and cookie-banner removal

    • Remote
    • Freemium
    • Web search and scraping
  19. Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  20. Research African companies from an assistant, with cited answers and spend approval

    Not checked yet
    • 2 tools
    • Remote
    • Freemium
    • Sales and CRM
  21. Test whether GPTBot, ClaudeBot, PerplexityBot and 10 more crawlers can reach a page

    • 1 tool
    • Remote
    • Free
    • Marketing and SEO
  22. Crawl and scrape sites to markdown, then ask what changed between runs

    Not checked yet
    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  23. Turn YouTube, TikTok, Instagram, Facebook, X and article URLs into transcripts and JSON

    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  24. Check whether a half-remembered thing can be found before your agent answers

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  25. Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  26. Web capture, MCP and x402 inspectors and a verified directory for agent tools

    • 34 tools
    • Remote
    • Freemium
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.