Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,168 MCP servers, 10 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, firecrawl and Firecrawl MCP Server. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 592

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  3. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 112K GitHub stars
    • AI models and LLM tooling
  4. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  5. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  6. Browser automation CLI that drives your logged-in Chrome, with reusable site commands

    • Open source
    • Open-source agent
    • 31.1K GitHub stars
    • Browser automation
  7. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  8. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  9. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  10. Guides Qdrant monitoring setup including Prometheus scraping, health probes, Hybrid Cloud metrics, alerting, and log centralization

    • Open source
    • 41.3K GitHub stars
    • Knowledge and memory
  11. Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs

    • Open source
    • 41.3K GitHub stars
    • Web search and scraping
  12. Bing Webmaster Tools + IndexNow extension

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  13. Full-site crawling, scraping, and site mapping via Firecrawl MCP

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  14. SEO geo

    Skill

    Audit and improve content for AI Overviews and answer engines, including citability, entity clarity, crawler access, brand signals, and pass

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  15. Web API for agents: scrape, search, cloud browsers and task agents, billed in credits

    • Freemium
    • Open-source agent
    • 745 GitHub stars
    • Web search and scraping
  16. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  17. Use when the user wants to fact-check an article or verify factual claims in a document

    • Open source
    • 770 GitHub stars
    • Web search and scraping
  18. Generates Puppeteer scripts for browser automation, scraping, and PDF generation

    • Open source
    • 392 GitHub stars
    • Browser automation
  19. Guides Qdrant monitoring setup including Prometheus scraping, Datadog integration, health probes, Hybrid Cloud metrics, alerting, and log ce

    • Open source
    • 269 GitHub stars
    • Knowledge and memory
  20. Multi-level research: built-in knowledge → web search → Tavily deep research API

    • Open source
    • 107 GitHub stars
    • Web search and scraping
  21. Use Exa for semantic/neural web search

    • Open source
    • 72 GitHub stars
    • Web search and scraping
  22. NeuralVerge

    MCP server

    Structured, source-tagged business and people data from 300+ sources for agents

    • 200 tools
    • Remote
    • Free trial
    • 1 GitHub star
    • Web search and scraping
  23. Search AI tools, agents and MCP servers and get install configs from inside your AI assistant.

    • 4 tools
    • Remote
    • Free
    • Developer tools
  24. easyserp

    MCP server

    Structured Google and Bing search results for agents, with recoverable requests

    Runs locally
    • Free trial
    • Web search and scraping
  25. ChangePing

    MCP server

    Watch web pages from Claude or ChatGPT and read summaries of what changed

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  26. ClawGrab

    MCP server

    Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  27. Pay-per-call tools for chain data, sanctions screening, web reading and PDFs

    • 15 tools
    • Remote
    • Freemium
    • Web search and scraping
  28. llms.ink

    MCP server

    Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  29. Search a catalogue of Apify Actors and n8n recipes for web-data jobs

    • 8 tools
    • Remote
    • Free
    • Web search and scraping