Skip to content

AI for web search and scraping

108 AI tools, agents, MCP servers and skills for web search and scraping, with pricing, platforms and alternatives for each.

Showing 1–30 of 108

  1. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  2. Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

    • Freemium
    • Open-source agent
    • 87.5K GitHub stars
    • Web search and scraping
  3. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  4. Pay-per-call tools for chain data, sanctions screening, web reading and PDFs

    • 15 tools
    • Remote
    • Freemium
    • Web search and scraping
  5. Unique.domains

    MCP server

    Look up domain status, registry records, registrar prices and the day's auctions

    • 51 tools
    • Remote
    • Freemium
    • Analytics and data
  6. Browserless

    MCP server

    Give an assistant a cloud browser to scrape, search, crawl and audit pages

    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  7. 31 web scraping, search and research tools that return clean Markdown or JSON

    Runs locally
    • 28 tools
    • Freemium
    • Web search and scraping
  8. FetchLayer

    MCP server

    Live data from Reddit, X, TikTok, app stores, ad libraries and 30+ more platforms

    • Remote
    • Freemium
    • Web search and scraping
  9. DataFluxor

    MCP server

    Remote MCP server for live Google results, SEO research and web scraping with credits

    • Remote
    • Freemium
    • Marketing and SEO
  10. Distill MCP

    MCP server

    Fetch web pages as clean Markdown for agents, honouring robots.txt and noai

    Runs locally
    • 2 tools
    • Freemium
    • Web search and scraping
  11. SerpApi

    MCP server

    Official SerpApi server: Google, Bing and other search results as JSON or Markdown

    • 4 tools
    • Remote
    • Freemium
    • Web search and scraping
  12. CRW Web Scraper

    MCP server

    Open-source web scraper for agents with scrape, crawl, map and search tools

    Runs locally
    • 3 tools
    • Freemium
    • Web search and scraping
  13. SnapRender

    MCP server

    Local MCP server for SnapRender screenshots, with a free trial key

    Runs locally
    • 3 tools
    • Freemium
    • Browser automation
  14. Web search, URL extract and page render for agents, paid per call over x402 on Base

    Runs locally
    • 8 tools
    • Freemium
    • Web search and scraping
  15. Search arXiv and OpenAlex papers with citations and PDF links, plus clean URL extraction

    Not checked yet
    • 24 tools
    • Remote
    • Freemium
    • Health and science
  16. Google Maps style local business leads, domain OSINT and B2B company dossiers

    Not checked yet
    • 24 tools
    • Remote
    • Freemium
    • Maps and travel
  17. Wield

    MCP server

    Turns websites into typed JSON tool calls that agents can search, read, log in and act on

    Not checked yet
    • 5 tools
    • Remote
    • Freemium
    • Browser automation
  18. Octocrawl

    MCP server

    Web scraping, mapping and crawling MCP server that returns clean Markdown

    Not checked yet
    • 2 tools
    • Remote
    • Freemium
    • Web search and scraping
  19. Company enrichment, FMCSA carriers, app reviews, live ATS jobs and LatAm tenders via Apify

    Not checked yet
    • 5 tools
    • Remote
    • Freemium
    • Analytics and data
  20. Remote MCP endpoint for Olostep's search, scrape and crawl API

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  21. Socialscrape

    MCP server

    Social media profiles, posts, comments, search and transcripts across 29 platforms

    Not checked yet
    • 11 tools
    • Remote
    • Freemium
    • Web search and scraping
  22. API.SH

    MCP server

    Search, call and bill public web data endpoints from chat via API.SH

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  23. Diffbot

    MCP server

    Search the web, extract pages and query a knowledge graph through Diffbot's index

    Not checked yet
    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  24. Clawifi

    MCP server

    Web search, page and PDF reading, and passage extraction with sources for agents

    Not checked yet
    • 6 tools
    • Remote
    • Freemium
    • Web search and scraping
  25. Spicrawl

    MCP server

    Scrape any URL to Markdown or JSON, with JavaScript rendering and batch jobs

    Not checked yet
    • 7 tools
    • Remote
    • Freemium
    • Web search and scraping
  26. Crawlora MCP

    MCP server

    Hosted MCP exposing 4,684 structured web-data tools across 534 platforms, returned as JSON

    Not checked yet
    • Remote
    • Freemium
    • Web search and scraping
  27. Fetch Router

    MCP server

    Web fetch for agents that checks pages for blocks and empty shells, bills only on success

    Not checked yet
    • 1 tool
    • Remote
    • Freemium
    • Web search and scraping
  28. Shotlane

    MCP server

    Give your agent a real Chrome for screenshots, Markdown, PDFs and phone-size page checks

    Not checked yet
    • 5 tools
    • Remote
    • Freemium
    • Browser automation