
Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON
- Freemium
- Open-source agent
- 87.6K GitHub stars
- Web search and scraping

Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON
Turns public web pages into clean Markdown and typed IR for developers
Bing and DuckDuckGo search results with page text, pay-per-query, for AI agents and RAG
Converts web pages or whole sites to LLM-ready Markdown for RAG pipelines and AI agents
Rust MCP server that cleans and reranks web pages into compact text for AI agents
Not checked yetMCP server that pulls Reddit posts, comments and buyer-intent threads via Apify
MCP server that crawls websites into LLM-ready Markdown and RAG chunks via Apify
Pulls timestamped YouTube transcripts and RAG chunks for AI pipelines via Apify
Web search and content extraction API that feeds live, structured results to AI agents
MCP server giving AI clients YouTube transcripts, metadata, search, comments and channel
Converts web pages into clean Markdown or plain text for LLMs, RAG and agents
Converts public web pages to clean Markdown or text for LLM and RAG pipelines
Turn YouTube captions, PDFs and web pages into clean Markdown for RAG
Turn audio, images, PDFs, video and web pages into text an agent can read
Pay-per-call URL-to-Markdown, B2B enrichment and bounty feeds for agents via x402
Hosted and open-source web scraper that returns clean Markdown with change tracking
Fetch web pages as clean Markdown for agents, honouring robots.txt and noai
Runs locallyWeb scraping, mapping and crawling MCP server that returns clean Markdown
Not checked yet
Remote MCP endpoint for Olostep's search, scrape and crawl API
Not checked yetFetch YouTube transcripts for agents on cloud servers through an Apify Actor
Runs locallyWeb search, page and PDF reading, and passage extraction with sources for agents
Crawl a website into clean Markdown, RAG chunks and llms.txt files
YouTube transcripts, search and summaries for Claude, Cursor and Codex
Turn pages or whole sites into clean Markdown or JSON for LLM and RAG pipelines
YouTube transcripts, metadata, search and comments for AI agents over MCP

Web search, Google results, page-to-markdown extraction and cited answers for agents
YouTube transcripts, metadata, comments, search and channel feeds, read only
Crawl a site into clean, main-content Markdown for RAG, with linked documents converted
Scrape pages and map sites from an agent, processed on EU servers
Runs locallyLive web search with quoted evidence and source authority tiers for agents
The most popular AI tools and MCP servers for rag on AI-Powered right now are crawl4ai, Atlas, Web Search API, Website to Markdown and Webshift. Rankings combine upvotes, recent activity and, for open-source projects, GitHub stars.
Yes. Of the 66 AI tools and MCP servers for rag listed, 8 are free or open source and 32 have a free plan. Use the Pricing filter to see only those.
AI-Powered lists 66 AI tools and MCP servers for rag. New listings are added daily from the official MCP Registry, GitHub and maker submissions.