Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,212 MCP servers, 15 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 189

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 203K GitHub stars
    • Web search and scraping
  3. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 113K GitHub stars
    • AI models and LLM tooling
  4. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  5. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.9K GitHub stars
    • Web search and scraping
  6. Browser automation CLI that drives your logged-in Chrome, with reusable site commands

    • Open source
    • Open-source agent
    • 31.1K GitHub stars
    • Browser automation
  7. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.8K GitHub stars
    • Research
  8. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  9. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  10. Pay-per-call tools for chain data, sanctions screening, web reading and PDFs

    • 15 tools
    • Remote
    • Freemium
    • Web search and scraping
  11. llms.ink

    MCP server

    Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  12. CheckAnySite

    MCP server

    Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  13. Unique.domains

    MCP server

    Look up domain status, registry records, registrar prices and the day's auctions

    • 51 tools
    • Remote
    • Freemium
    • Analytics and data
  14. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  15. Browserless

    MCP server

    Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  16. ResearchZosho

    MCP server

    Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  17. 31 web scraping, search and research tools that return clean Markdown or JSON

    Runs locally
    • 28 tools
    • Freemium
    • Web search and scraping
  18. FetchLayer

    MCP server

    Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  19. DataFluxor

    MCP server

    Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  20. Distill MCP

    MCP server

    Fetch web pages as clean Markdown for agents, honouring robots.txt and noai

    Runs locally
    • 2 tools
    • Freemium
    • Web search and scraping
  21. proxy-scraper

    MCP server

    Hourly-checked free HTTP and SOCKS proxy lists with a CLI and MCP tools to fetch pages

    Runs locally
    • Free
    • Web search and scraping
  22. SerpApi

    MCP server

    Official SerpApi server: Google, Bing and other search results as JSON or Markdown

    • 4 tools
    • Remote
    • Freemium
    • Web search and scraping
  23. Fetch web pages as markdown with SSRF protection, reader mode, feeds and robots checks

    Runs locally
    • 15 tools
    • Open source
    • Web search and scraping
  24. CRW Web Scraper

    MCP server

    Open-source web scraper for agents with scrape, crawl, map and search tools

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  25. SnapRender

    MCP server

    Local MCP server for SnapRender screenshots, with a free trial key

    Runs locally
    • 3 tools
    • Freemium
    • Browser automation
  26. Web search, URL extract and page render for agents, paid per call over x402 on Base

    Runs locally
    • 8 tools
    • Freemium
    • Web search and scraping
  27. Search arXiv and OpenAlex papers with citations and PDF links, plus clean URL extraction

    Not checked yet
    • 24 tools
    • Remote
    • Freemium
    • Health and science