Skip to content

AI for web search and scraping

237 AI tools, agents and MCP servers for web search and scraping, with pricing, platforms and alternatives for each.

Showing 1–30 of 237

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  3. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 112K GitHub stars
    • AI models and LLM tooling
  4. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  5. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  6. Browser automation CLI that drives your logged-in Chrome, with reusable site commands

    • Open source
    • Open-source agent
    • 31.1K GitHub stars
    • Browser automation
  7. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  8. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  9. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  10. EuroScrape

    MCP server

    European flight, hotel, power-price, marketplace and company data tools for agents

    Runs locally
    • 13 tools
    • Paid
    • Web search and scraping
  11. easyserp

    MCP server

    Structured Google and Bing search results for agents, with recoverable requests

    Runs locally
    • Free trial
    • Web search and scraping
  12. Pay-per-call tools for chain data, sanctions screening, web reading and PDFs

    • 15 tools
    • Remote
    • Freemium
    • Web search and scraping
  13. llms.ink

    MCP server

    Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  14. Fetch pages and search the web through a Singapore 4G/5G mobile IP from your agent

    • 4 tools
    • Remote
    • Free trial
    • Web search and scraping
  15. maviapi

    MCP server

    Live JSON from sites with no API, plus agent-built APIs for any website

    Runs locally
    • 9 tools
    • Free trial
    • Web search and scraping
  16. CheckAnySite

    MCP server

    Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  17. Unique.domains

    MCP server

    Look up domain status, registry records, registrar prices and the day's auctions

    • 51 tools
    • Remote
    • Freemium
    • Analytics and data
  18. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  19. Hosted and open-source web scraper that returns clean Markdown with change tracking

    • Remote
    • Free trial
    • Web search and scraping
  20. Browserless

    MCP server

    Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  21. ResearchZosho

    MCP server

    Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  22. 31 web scraping, search and research tools that return clean Markdown or JSON

    Runs locally
    • 28 tools
    • Freemium
    • Web search and scraping
  23. Novada MCP

    MCP server

    One MCP server for web search, page extraction, crawling and platform scraping

    • 9 tools
    • Remote
    • Free trial
    • Web search and scraping
  24. FetchLayer

    MCP server

    Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  25. DataFluxor

    MCP server

    Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  26. okdata

    MCP server

    Pay-per-call social data from Douyin, Xiaohongshu, TikTok, X, YouTube and more

    • 5 tools
    • Remote
    • Paid
    • Web search and scraping