Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,212 MCP servers, 15 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 84

  1. Open-source research agent that writes cited reports from the web and your documents

    • Open source
    • Open-source agent
    • 30.8K GitHub stars
    • Research
  2. Browser engine and CLI built for agents that browse, extract and crawl at scale

    • Open source
    • Open-source agent
    • 1.2K GitHub stars
    • Browser automation
  3. Self-hosted browser agent that scrapes protected sites from a plain-English prompt

    • Open source
    • Open-source agent
    • 180 GitHub stars
    • Web search and scraping
  4. llms.ink

    MCP server

    Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  5. Search a catalogue of Apify Actors and n8n recipes for web-data jobs

    • 8 tools
    • Remote
    • Free
    • Web search and scraping
  6. Scraper Finder

    MCP server

    Look up the official API and ranked scraping tools for 172 websites

    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  7. x-toolkit

    MCP server

    Read public X Articles, video transcripts and posts as plain text, no account needed

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  8. CheckAnySite

    MCP server

    Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  9. Search a free directory of MCP servers and x402 services, or list your own

    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  10. Free check of whether GPTBot, ClaudeBot and PerplexityBot can really reach a website

    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  11. Test whether GPTBot, ClaudeBot, PerplexityBot and 10 more crawlers can reach a page

    • 1 tool
    • Remote
    • Free
    • Marketing and SEO
  12. Check whether a half-remembered thing can be found before your agent answers

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  13. proxy-scraper

    MCP server

    Hourly-checked free HTTP and SOCKS proxy lists with a CLI and MCP tools to fetch pages

    Runs locally
    • Free
    • Web search and scraping
  14. Copy Markdown

    MCP server

    Turn any web page into clean Markdown, design tokens and a link list for LLMs

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Files and documents
  15. Vouch

    MCP server

    Check that links load and quotes really appear on a page, from your assistant

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Developer tools
  16. Find page-extraction services from a frozen catalogue; a time-limited free experiment

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Web search and scraping
  17. Agents Allowed

    MCP server

    Free lookup an agent can query to see if a site permits AI agents to browse, shop or book

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  18. Search immunology and virology literature, trials and grants across six databases at once

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Health and science
  19. Watch

    MCP server

    Watch a web page, CSS selector, JSON value or price and get changes by poll or webhook

    Not checked yet
    • 6 tools
    • Remote
    • Free
    • Web search and scraping
  20. Search expiring and recently dropped domain names by drop date, rank and linking domains

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Marketing and SEO
  21. Tracert

    MCP server

    Routes other agents mapped for web tasks, so yours verify instead of re-researching

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  22. Search 1,500+ indie-made products as alternatives to big-name software from your AI

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  23. Remote MCP server that searches the MCP Harbor directory of MCP servers and agent tools

    Not checked yet
    • 2 tools
    • Remote
    • Open source
    • Developer tools
  24. Search a public catalogue of Telegram channels, groups, bots and listings, no key needed

    Not checked yet
    • 1 tool
    • Remote
    • Free
    • Communication
  25. Query your website change-tracking history and monitors in plain language

    Not checked yet
    • Remote
    • Open source
    • Productivity
  26. LastSearch

    MCP server

    Web research for agents that returns cited claims with confidence scores

    Runs locally
    • Open source
    • Web search and scraping
  27. Live news risk radars, market sensors and an AI anchor for agents over MCP

    Not checked yet
    • 12 tools
    • Remote
    • Free
    • Analytics and data
  28. MakerRadar

    MCP server

    Search launched products, alternatives and weekly launch boards on MakerRadar

    Not checked yet
    • 14 tools
    • Remote
    • Free
    • Web search and scraping