Crawlberg vs XERJ
Crawlberg is an open-source MCP server for web search and scraping. XERJ is an open-source MCP server for knowledge and memory.
- Crawlberg is best for feed a rag pipeline with clean site content.
- XERJ is best for search local code and docs from an agent.
- What it is
- Crawlberg is an open-source MCP server for web search and scraping.XERJ is an open-source MCP server for knowledge and memory.
- Best for
- Feed a RAG pipeline with clean site contentSearch local code and docs from an agent
- Kind
- MCP serverMCP server
- Pricing
- Open sourceOpen source
- Platforms
- CLI, self-hosted, API, Linux, macOS, WindowsCLI, Linux, macOS, self-hosted, API
- Runs
- LocallyLocally
- GitHub stars
- 19.2K
- Last commit
- Nov 2026
- License
- Apache-2.0
- Open source
- Good for
- Feed a RAG pipeline with clean site content; Map a website's link structure and sitemap; Archive documentation as MarkdownSearch local code and docs from an agent; Give an agent persistent memory notes; Look up real API definitions
What Crawlberg does
Single-page or site crawl
Scrape one URL or crawl a whole site with configurable depth, page limits and concurrency.
Markdown conversion
Pages become clean Markdown with citations and document structure.
Rich metadata
Extracts titles, links, images, JSON-LD, Open Graph, hreflang and headings.
SSRF protection
Refuses private, loopback and cloud-metadata addresses unless you opt out.
Browser fallback
Uses an optional headless browser for JavaScript-heavy single-page apps.
Many language bindings
Same core exposed to Python, Node.js, Rust, Go, Java, Ruby and others.
What XERJ does
Single binary
Search, vectors, facts and memory in one Rust program.
Evidence-backed answers
Answers come with the program and evidence used.
Runs locally
Works offline on a CPU, on a laptop or server.
Elasticsearch-compatible API
Exposes an API compatible with Elasticsearch.
Reference coding
Index open-source repos so agents consult real definitions.
MCP integration
Stdio MCP server connects to a running node.
Crawlberg vs XERJ: questions
Is Crawlberg or XERJ cheaper?
Both are open source. Compare current plans on each official site.
What is the difference between Crawlberg and XERJ?
Crawlberg is an open-source MCP server for web search and scraping. XERJ is an open-source MCP server for knowledge and memory.
