Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,168 MCP servers, 13 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 192

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 112K GitHub stars
    • AI models and LLM tooling
  3. Browser automation CLI that drives your logged-in Chrome, with reusable site commands

    • Open source
    • Open-source agent
    • 31.1K GitHub stars
    • Browser automation
  4. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  5. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  6. Researches competitors into dated, source-backed profile files using scraping and SEO data

    • Open source
    • 60.9K GitHub stars
    • Sales and CRM
  7. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  8. Guides Qdrant monitoring setup including Prometheus scraping, health probes, Hybrid Cloud metrics, alerting, and log centralization

    • Open source
    • 41.3K GitHub stars
    • Knowledge and memory
  9. Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs

    • Open source
    • 41.3K GitHub stars
    • Web search and scraping
  10. Adds Bing Webmaster link data and IndexNow URL submission to claude-seo

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  11. Live SERP, keyword, backlink and AI-mention data from DataForSEO inside claude-seo

    • Open source
    • 30.2K GitHub stars
    • Marketing and SEO
  12. Crawls, maps and scrapes whole sites for SEO audits through the Firecrawl MCP server

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  13. SEO geo

    Skill

    Audits pages for citability in AI Overviews, ChatGPT, Perplexity and other answer engines

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  14. Research five direct competitors and recommend how to position against them

    • Open source
    • 28.1K GitHub stars
    • Marketing and SEO
  15. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  16. Use when the user wants to fact-check an article or verify factual claims in a document

    • Open source
    • 770 GitHub stars
    • Web search and scraping
  17. Generates Puppeteer scripts for browser automation, scraping, and PDF generation

    • Open source
    • 392 GitHub stars
    • Browser automation
  18. Guides Qdrant monitoring setup including Prometheus scraping, Datadog integration, health probes, Hybrid Cloud metrics, alerting, and log ce

    • Open source
    • 269 GitHub stars
    • Knowledge and memory
  19. Multi-level research: built-in knowledge → web search → Tavily deep research API

    • Open source
    • 107 GitHub stars
    • Web search and scraping
  20. Use Exa for semantic/neural web search

    • Open source
    • 72 GitHub stars
    • Web search and scraping
  21. NeuralVerge

    MCP server

    Structured, source-tagged business and people data from 300+ sources for agents

    • 200 tools
    • Remote
    • Free trial
    • 1 GitHub star
    • Web search and scraping
  22. easyserp

    MCP server

    Structured Google and Bing search results for agents, with recoverable requests

    Runs locally
    • Free trial
    • Web search and scraping
  23. Fetch pages and search the web through a Singapore 4G/5G mobile IP from your agent

    • 4 tools
    • Remote
    • Free trial
    • Web search and scraping
  24. maviapi

    MCP server

    Live JSON from sites with no API, plus agent-built APIs for any website

    Runs locally
    • 9 tools
    • Free trial
    • Web search and scraping
  25. Search Console clicks, positions, URL inspection and sitemaps for your AI agent

    Runs locally
    • 8 tools
    • Open source
    • Web search and scraping
  26. Hosted and open-source web scraper that returns clean Markdown with change tracking

    • Remote
    • Free trial
    • Web search and scraping
  27. ResearchZosho

    MCP server

    Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  28. Novada MCP

    MCP server

    One MCP server for web search, page extraction, crawling and platform scraping

    • 9 tools
    • Remote
    • Free trial
    • Web search and scraping
  29. Fetch web pages as markdown with SSRF protection, reader mode, feeds and robots checks

    Runs locally
    • 15 tools
    • Open source
    • Web search and scraping
  30. Self-hosted Google Search Console MCP for analytics, URL inspection and sitemaps

    Not checked yet
    • Remote
    • Open source
    • Marketing and SEO