Skip to content

AI for web search and scraping

194 AI tools, agents, MCP servers and skills for web search and scraping, with pricing, platforms and alternatives for each.

Showing 1–30 of 194

  1. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  2. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  3. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  4. Web API for agents: scrape, search, cloud browsers and task agents, billed in credits

    • Freemium
    • Open-source agent
    • 745 GitHub stars
    • Web search and scraping
  5. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  6. ChangePing

    MCP server

    Watch web pages from Claude or ChatGPT and read summaries of what changed

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  7. ClawGrab

    MCP server

    Fetch transcripts of public YouTube, TikTok, Vimeo and podcast links for AI use

    Runs locally
    • 6 tools
    • Freemium
    • Web search and scraping
  8. Turn YouTube captions, PDFs and web pages into clean Markdown for RAG

    • Remote
    • Freemium
    • Knowledge and memory
  9. Read Bluesky, Telegram, note.com and WeChat posts without logging in

    • Remote
    • Freemium
    • Web search and scraping
  10. Check which e-commerce stores publish a UCP profile and what it declares

    • Remote
    • Freemium
    • E-commerce
  11. YouTube transcripts, comments, video stats and channel monitoring for AI agents

    • 6 tools
    • Remote
    • Freemium
    • Design and media
  12. Paid web, OpenAPI and Base chain evidence for agents, billed per call in USDC

    • 12 tools
    • Remote
    • Freemium
    • Web search and scraping
  13. Browserless

    MCP server

    Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  14. ResearchZosho

    MCP server

    Local deep research assistant that checks each citation and keeps what it finds

    Runs locally
    • Open source
    • Knowledge and memory
  15. Search what was said on YouTube and podcasts and get quotes cued to the second

    • 4 tools
    • Remote
    • Freemium
    • Design and media
  16. SpecStream

    MCP server

    Web search, page extraction and structured live-web datasets for agents

    • 3 tools
    • Remote
    • Freemium
    • Web search and scraping
  17. Site-Shot

    MCP server

    Website screenshots for AI agents: full page, ad and cookie-banner removal

    • Remote
    • Freemium
    • Web search and scraping
  18. QuietDiff

    MCP server

    Watch chosen fields on web pages and have an assistant create monitors and read changes

    • 9 tools
    • Remote
    • Freemium
    • Web search and scraping
  19. FetchLayer

    MCP server

    Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  20. Konexium

    MCP server

    Research African companies from an assistant, with cited answers and spend approval

    Not checked yet
    • 2 tools
    • Remote
    • Freemium
    • Sales and CRM
  21. MeshArc

    MCP server

    Crawl and scrape sites to markdown, then ask what changed between runs

    Not checked yet
    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  22. Post Reef

    MCP server

    Turn YouTube, TikTok, Instagram, Facebook, X and article URLs into transcripts and JSON

    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  23. DataFluxor

    MCP server

    Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  24. PageVerified

    MCP server

    Web capture, MCP and x402 inspectors and a verified directory for agent tools

    • 34 tools
    • Remote
    • Freemium
    • Web search and scraping
  25. Export public Bluesky posts by keyword or handle without a login, as JSON or CSV

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping