Skip to content

Crawlberg vs XERJ

Crawlberg is an open-source MCP server for web search and scraping. XERJ is an open-source MCP server for knowledge and memory.

  • Crawlberg is best for feed a rag pipeline with clean site content.
  • XERJ is best for search local code and docs from an agent.
What it is
Crawlberg is an open-source MCP server for web search and scraping.
XERJ is an open-source MCP server for knowledge and memory.
Best for
Feed a RAG pipeline with clean site content
Search local code and docs from an agent
Kind
MCP server
MCP server
Pricing
Open source
Open source
Platforms
CLI, self-hosted, API, Linux, macOS, Windows
CLI, Linux, macOS, self-hosted, API
Runs
Locally
Locally
GitHub stars
19.2K
Last commit
Nov 2026
License
Apache-2.0
Open source
Good for
Feed a RAG pipeline with clean site content; Map a website's link structure and sitemap; Archive documentation as Markdown
Search local code and docs from an agent; Give an agent persistent memory notes; Look up real API definitions

What Crawlberg does

  • Single-page or site crawl

    Scrape one URL or crawl a whole site with configurable depth, page limits and concurrency.

  • Markdown conversion

    Pages become clean Markdown with citations and document structure.

  • Rich metadata

    Extracts titles, links, images, JSON-LD, Open Graph, hreflang and headings.

  • SSRF protection

    Refuses private, loopback and cloud-metadata addresses unless you opt out.

  • Browser fallback

    Uses an optional headless browser for JavaScript-heavy single-page apps.

  • Many language bindings

    Same core exposed to Python, Node.js, Rust, Go, Java, Ruby and others.

What XERJ does

  • Single binary

    Search, vectors, facts and memory in one Rust program.

  • Evidence-backed answers

    Answers come with the program and evidence used.

  • Runs locally

    Works offline on a CPU, on a laptop or server.

  • Elasticsearch-compatible API

    Exposes an API compatible with Elasticsearch.

  • Reference coding

    Index open-source repos so agents consult real definitions.

  • MCP integration

    Stdio MCP server connects to a running node.

Crawlberg vs XERJ: questions

Is Crawlberg or XERJ cheaper?

Both are open source. Compare current plans on each official site.

What is the difference between Crawlberg and XERJ?

Crawlberg is an open-source MCP server for web search and scraping. XERJ is an open-source MCP server for knowledge and memory.