Skip to content

AI for web search and scraping

201 AI tools, agents, MCP servers and skills for web search and scraping, with pricing, platforms and alternatives for each.

Showing 1–30 of 201

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  3. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 112K GitHub stars
    • AI models and LLM tooling
  4. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  5. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  6. Browser automation CLI that drives your logged-in Chrome, with reusable site commands

    • Open source
    • Open-source agent
    • 31.1K GitHub stars
    • Browser automation
  7. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  8. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  9. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  10. EuroScrape

    MCP server

    European flight, hotel, power-price, marketplace and company data tools for agents

    Runs locally
    • 13 tools
    • Paid
    • Web search and scraping
  11. Pay-per-call tools for chain data, sanctions screening, web reading and PDFs

    • 15 tools
    • Remote
    • Freemium
    • Web search and scraping
  12. Unique.domains

    MCP server

    Look up domain status, registry records, registrar prices and the day's auctions

    • 51 tools
    • Remote
    • Freemium
    • Analytics and data
  13. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  14. Browserless

    MCP server

    Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  15. ResearchZosho

    MCP server

    Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  16. 31 web scraping, search and research tools that return clean Markdown or JSON

    Runs locally
    • 28 tools
    • Freemium
    • Web search and scraping
  17. FetchLayer

    MCP server

    Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  18. DataFluxor

    MCP server

    Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  19. okdata

    MCP server

    Pay-per-call social data from Douyin, Xiaohongshu, TikTok, X, YouTube and more

    • 5 tools
    • Remote
    • Paid
    • Web search and scraping
  20. Distill MCP

    MCP server

    Fetch web pages as clean Markdown for agents, honouring robots.txt and noai

    Runs locally
    • 2 tools
    • Freemium
    • Web search and scraping
  21. SerpApi

    MCP server

    Official SerpApi server: Google, Bing and other search results as JSON or Markdown

    • 4 tools
    • Remote
    • Freemium
    • Web search and scraping
  22. RedditAPIs

    MCP server

    Reddit search, comments, users and monitors as 61 pay-per-call MCP tools

    Not checked yet
    • 1 tool
    • Remote
    • Paid
    • Web search and scraping
  23. Fetch web pages as markdown with SSRF protection, reader mode, feeds and robots checks

    Runs locally
    • 15 tools
    • Open source
    • Web search and scraping
  24. CRW Web Scraper

    MCP server

    Open-source web scraper for agents with scrape, crawl, map and search tools

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  25. SnapRender

    MCP server

    Local MCP server for SnapRender screenshots, with a free trial key

    Runs locally
    • 3 tools
    • Freemium
    • Browser automation
  26. Web search, URL extract and page render for agents, paid per call over x402 on Base

    Runs locally
    • 8 tools
    • Freemium
    • Web search and scraping
  27. Search arXiv and OpenAlex papers with citations and PDF links, plus clean URL extraction

    Not checked yet
    • 24 tools
    • Remote
    • Freemium
    • Health and science