Skip to content

Agent skills for web search and scraping

15 agent skills that give AI live web search, page fetching and structured data extraction. Each one shows what it can run and how to install it in Claude Code, Codex, Cursor and other agents.
  1. Researches competitors into dated, source-backed profile files using scraping and SEO data

    • Open source
    • 60.9K GitHub stars
    • Sales and CRM
  2. Adds Bing Webmaster link data and IndexNow URL submission to claude-seo

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  3. Live SERP, keyword, backlink and AI-mention data from DataForSEO inside claude-seo

    • Open source
    • 30.2K GitHub stars
    • Marketing and SEO
  4. Crawls, maps and scrapes whole sites for SEO audits through the Firecrawl MCP server

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  5. Audits pages for citability in AI Overviews, ChatGPT, Perplexity and other answer engines

    • Open source
    • 30.2K GitHub stars
    • Web search and scraping
  6. Research five direct competitors and recommend how to position against them

    • Open source
    • 28.1K GitHub stars
    • Marketing and SEO
  7. Capture public webpages as local PNG, JPEG or PDF files via the Latchshot service

    • Open source
    • 41.3K GitHub stars
    • Web search and scraping
  8. Guides setting up Prometheus scraping, health probes, alerting and logs for Qdrant

    • Open source
    • 41.3K GitHub stars
    • Knowledge and memory
  9. Guides an agent through Xquik X API integrations for tweet search, lookups and exports

    • Open source
    • 41.3K GitHub stars
    • Web search and scraping
  10. Lets your agent write .NET code for Azure Maps geocoding, routing, tiles, weather and IP

    • Open source
    • 3.2K GitHub stars
    • Maps and travel
  11. Use when the user wants to fact-check an article or verify factual claims in a document

    • Open source
    • 770 GitHub stars
    • Web search and scraping
  12. Generates Puppeteer scripts for browser automation, scraping, and PDF generation

    • Open source
    • 392 GitHub stars
    • Browser automation
  13. Guides Qdrant monitoring setup including Prometheus scraping, Datadog integration, health probes, Hybrid Cloud metrics, alerting, and log ce

    • Open source
    • 269 GitHub stars
    • Knowledge and memory
  14. Multi-level research: built-in knowledge → web search → Tavily deep research API

    • Open source
    • 107 GitHub stars
    • Web search and scraping
  15. Use Exa for semantic/neural web search

    • Open source
    • 72 GitHub stars
    • Web search and scraping

See every kind of AI for web search and scraping: tools, agents, MCP servers and skills.Ranked: the best AI for web search and scraping.