Skip to content

AI tools, agents, MCP servers and skills for web search and scraping

1,212 MCP servers, 15 agent skills, 6 AI tools and 2 AI agents for web search and scraping, including Scrapling MCP Server, Competitor profiling and firecrawl. Each listing shows pricing, platforms and alternatives; use the kind links to see only one kind.

Showing 1–30 of 59

  1. llms.ink

    MCP server

    Turn any website, GitHub repo, PDF or video link into llms.txt or llms-full.txt

    • 7 tools
    • Remote
    • Free
    • AI models and LLM tooling
  2. x-toolkit

    MCP server

    Read public X Articles, video transcripts and posts as plain text, no account needed

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  3. CheckAnySite

    MCP server

    Ask an agent what a website is: AI builder traces, tech stack, traffic rank and trust

    • 6 tools
    • Remote
    • Free
    • Analytics and data
  4. Search a free directory of MCP servers and x402 services, or list your own

    • 12 tools
    • Remote
    • Free
    • Web search and scraping
  5. Free check of whether GPTBot, ClaudeBot and PerplexityBot can really reach a website

    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  6. Test whether GPTBot, ClaudeBot, PerplexityBot and 10 more crawlers can reach a page

    • 1 tool
    • Remote
    • Free
    • Marketing and SEO
  7. Check whether a half-remembered thing can be found before your agent answers

    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  8. Copy Markdown

    MCP server

    Turn any web page into clean Markdown, design tokens and a link list for LLMs

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Files and documents
  9. Vouch

    MCP server

    Check that links load and quotes really appear on a page, from your assistant

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Developer tools
  10. Keryx

    MCP server

    Cited research answers with source budgets and visible USDC payment states

    Not checked yet
    • 3 tools
    • Remote
    • Free trial
    • Web search and scraping
  11. Agents Allowed

    MCP server

    Free lookup an agent can query to see if a site permits AI agents to browse, shop or book

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  12. Social Data API

    MCP server

    One API key for public TikTok data, with X, Instagram and Reddit planned

    Not checked yet
    • Remote
    • Free trial
    • Analytics and data
  13. Search immunology and virology literature, trials and grants across six databases at once

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Health and science
  14. Watch

    MCP server

    Watch a web page, CSS selector, JSON value or price and get changes by poll or webhook

    Not checked yet
    • 6 tools
    • Remote
    • Free
    • Web search and scraping
  15. sociallinks.dev

    MCP server

    Find a creator's other social accounts from one profile link, with confidence scores

    Not checked yet
    • 4 tools
    • Remote
    • Free trial
    • Marketing and SEO
  16. mowteng

    MCP server

    Google Search Console, Bing and Yandex data inside your AI assistant, with SEO analyses

    Not checked yet
    • 16 tools
    • Remote
    • Free trial
    • Web search and scraping
  17. Search 1,500+ indie-made products as alternatives to big-name software from your AI

    Not checked yet
    • 4 tools
    • Remote
    • Free
    • Web search and scraping
  18. Search a public catalogue of Telegram channels, groups, bots and listings, no key needed

    Not checked yet
    • 1 tool
    • Remote
    • Free
    • Communication
  19. Rivul

    MCP server

    Search 300M+ scholarly paper records from Claude, Cursor or Codex, free

    Not checked yet
    • 2 tools
    • Remote
    • Free
    • Health and science
  20. Rankvisit

    MCP server

    MCP server that lets Claude or ChatGPT read Search Console, Bing and Fathom data

    Not checked yet
    • Remote
    • Free trial
    • Web search and scraping
  21. Ahel

    MCP server

    Pay-per-call data for agents: company, people, SEO, social and EU register lookups

    Not checked yet
    • Remote
    • Free trial
    • Sales and CRM
  22. WeProxy

    MCP server

    Turkey-focused proxy catalogue: product picks, pricing, coverage, error diagnosis

    Not checked yet
    • 7 tools
    • Remote
    • Free
    • Web search and scraping
  23. Nymwell Lookup

    MCP server

    Check domain availability and registrar prices and look up public domain records

    Not checked yet
    • 3 tools
    • Remote
    • Free
    • Web search and scraping
  24. Mamitsu MCP

    MCP server

    MCP server that pulls structured JSON from public Instagram, TikTok, YouTube and X posts

    Not checked yet
    • 1 tool
    • Remote
    • Free trial
    • Web search and scraping
  25. X Reader MCP

    MCP server

    Read public X posts and full X Articles as clean Markdown, with no X API key

    Not checked yet
    • 1 tool
    • Remote
    • Free
    • Web search and scraping
  26. Look up conference and journal deadlines, CCF and CORE rankings and acceptance rates

    Not checked yet
    • 8 tools
    • Remote
    • Free
    • Web search and scraping
  27. Facebook Data

    MCP server

    Read public Facebook Pages, groups, Marketplace and Ad Library ads with no login

    Not checked yet
    • 1 tool
    • Remote
    • Free trial
    • Marketing and SEO
  28. Instagram Data

    MCP server

    Read public Instagram profiles, posts, reels, hashtags and locations without logging in

    Not checked yet
    • 1 tool
    • Remote
    • Free trial
    • Marketing and SEO