Skip to content

AI for web search and scraping

15 AI tools, agents, MCP servers and skills for web search and scraping, with pricing, platforms and alternatives for each.

Showing 1–15 of 15

  1. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.7K GitHub stars
    • Research
  2. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  3. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  4. Remote MCP server that searches the MCP Harbor directory of MCP servers and agent tools

    Not checked yet
    • 2 tools
    • Remote
    • Open source
    • Developer tools
  5. Query your website change-tracking history and monitors in plain language

    Not checked yet
    • Remote
    • Open source
    • Productivity
  6. LastSearch

    MCP server

    Web research for agents that returns cited claims with confidence scores

    Runs locally
    • Open source
    • Web search and scraping
  7. Turn any web page into clean Markdown with one tool, run locally with no API key

    Runs locally
    • 1 tool
    • Open source
    • Files and documents
  8. searxNcrawl

    MCP server

    Open-source MCP server for web search via SearXNG or You.com and Markdown crawling

    Runs locally
    • 3 tools
    • Open source
    • Web search and scraping
  9. Crawlberg

    MCP server

    Open-source crawler that turns websites into Markdown and JSON, with a local MCP server

    Runs locally
    • Open source
    • Web search and scraping
  10. shortfree

    MCP server

    Checks live whether short .com, .ai, .io and other domains are free; suggests free ones

    Runs locally
    • Open source
    • Developer tools
  11. Honestkit MCP

    MCP server

    18 Apify-backed web data tools: WHOIS, DNS, SEO audits, app store ranks, company lookups

    Runs locally
    • 14 tools
    • Open source
    • Web search and scraping
  12. Check whether Chinese, Japanese and Korean page titles fit Google by pixel width

    Runs locally
    • 2 tools
    • Open source
    • Web search and scraping
  13. Keyless web, news, image, video and book search for AI agents, via MCP or REST

    Runs locally
    • Open source
    • Web search and scraping
  14. searchio

    MCP server

    Keyless web search, page reading and product lookup for agents, with a bot-wall ladder

    Runs locally
    • Open source
    • Web search and scraping
  15. RoamProxy

    MCP server

    Fetch pages and build proxy URLs through residential, datacenter and mobile IPs worldwide

    Runs locally
    • 4 tools
    • Open source
    • Web search and scraping