Skip to content

Agent skills for web search and scraping

8 agent skills that give AI live web search, page fetching and structured data extraction. Each one shows what it can run and how to install it in Claude Code, Codex, Cursor and other agents.
  1. Guides Qdrant monitoring setup including Prometheus scraping, health probes, Hybrid Cloud metrics, alerting, and log centralization

    • Open source
    • 41.3K GitHub stars
    • Knowledge and memory
  2. Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs

    • Open source
    • 41.3K GitHub stars
    • Web search and scraping
  3. Bing Webmaster Tools + IndexNow extension

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  4. Full-site crawling, scraping, and site mapping via Firecrawl MCP

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  5. Audit and improve content for AI Overviews and answer engines, including citability, entity clarity, crawler access, brand signals, and pass

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  6. Generates Puppeteer scripts for browser automation, scraping, and PDF generation

    • Open source
    • 392 GitHub stars
    • Browser automation
  7. Guides Qdrant monitoring setup including Prometheus scraping, Datadog integration, health probes, Hybrid Cloud metrics, alerting, and log ce

    • Open source
    • 269 GitHub stars
    • Knowledge and memory
  8. Use Exa for semantic/neural web search

    • Open source
    • 72 GitHub stars
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.