Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,212 MCP servers, 15 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 365

  1. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 203K GitHub stars
    • Web search and scraping
  2. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  3. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.8K GitHub stars
    • Research
  4. Web API for agents: scrape, search, cloud browsers and task agents, billed in credits

    • Freemium
    • Open-source agent
    • 746 GitHub stars
    • Web search and scraping
  5. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  6. API Pick

    MCP server

    27 pay-per-call search tools: web, news, SEC, patents, clinical and more

    • 27 tools
    • Remote
    • Paid
    • Web search and scraping
  7. ChangePing

    MCP server

    Watch web pages from Claude or ChatGPT and read summaries of what changed

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  8. ClawGrab

    MCP server

    Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  9. Turn YouTube captions, PDFs and web pages into clean Markdown for RAG

    • Remote
    • Freemium
    • Knowledge and memory
  10. Read Bluesky, Telegram, note.com and WeChat posts without logging in

    • Remote
    • Freemium
    • Web search and scraping
  11. Read public Telegram channel posts with views, reactions and media, no login

    • Remote
    • Paid
    • Communication
  12. Track EU TikTok Ad Library ads by advertiser or keyword, with change status

    • Remote
    • Paid
    • Marketing and SEO
  13. Check which e-commerce stores publish a UCP profile and what it declares

    • Remote
    • Freemium
    • E-commerce
  14. YouTube transcripts, comments, video stats and channel monitoring for AI agents

    • 6 tools
    • Remote
    • Freemium
    • Design and media
  15. Paid web, OpenAPI and Base chain evidence for agents, billed per call in USDC

    • 12 tools
    • Remote
    • Freemium
    • Web search and scraping