Skip to content
MCP server

Webshift

Rust MCP server that fetches, cleans and reranks web search results for AI agents

Pricing
Open source
Platforms
CLI, API, Linux
Version
0.2.14
Listing updated
today

Similar MCP servers

About Webshift

Webshift is an open-source MCP server for web search and scraping. It is available on CLI, API, Linux, macOS, Windows and self-hosted.

The README says WebShift is a Rust library and MCP server aimed at turning noisy web pages into compact text that fits within an LLM's context budget. It removes scripts, navigation, banners and similar clutter, strips problematic Unicode control characters, and then reranks the surviving pages with BM25 before applying size limits across all sources.

The MCP server ships as a single binary called mcp-webshift and exposes webshift_query, webshift_fetch and webshift_onboarding over stdio. Search can go through SearXNG (self-hosted, no key), Brave, Tavily, Exa, SerpAPI, Google, Bing or a generic HTTP backend. LLM query expansion, summarization and reranking are optional and disabled by default.

The project is aimed at developers building agents or Rust pipelines that need budget-controlled web text. The README is explicit that it parses static HTML only and does not run JavaScript, render pages or handle logins, so it is a poor fit for SPA-heavy sites or browser automation. It is still in beta.

Tools this server provides (3)

Taken from the project documentation; the running server may differ.

webshift_query
Full search pipeline: search, fetch, clean, rerank and optionally summarize.
webshift_fetch
Single page fetch and clean.
webshift_onboarding
Returns a JSON guide for the agent covering budgets, backends and tips.

What it does

Noise removal and cleanup
Strips scripts, navigation, ads and cookie banners from HTML and returns plain text.
BM25 reranking
Orders fetched pages with deterministic BM25 scoring, with optional LLM-assisted second-tier reranking.
Hard size budgets
Applies per-page character caps, total query budgets and streaming download limits.
Multiple search backends
Supports SearXNG, Brave, Tavily, Exa, SerpAPI, Google, Bing and a configurable HTTP backend.
Binary filtering
Skips PDFs, archives and executables before making any network request.
Rust library API
Exposes clean, fetch and query functions, plus text-map round-tripping of HTML text nodes.

Good for

  • Giving an AI agent web search results as clean, size-limited text
  • Cleaning raw HTML in a Rust data pipeline before passing it to an LLM
  • Translating or rewriting page text while keeping markup and links intact
  • Running a self-hosted web search tool for an MCP client without cloud API keys

Pricing

Open source

Free and MIT licensed; search backends such as Brave or Google may have their own paid tiers.

Webshift FAQ

Does WebShift render JavaScript-heavy pages?

No. The README says it parses static HTML and does not execute JavaScript.

Which search backend is free and needs no API key?

The README names SearXNG as free, self-hosted and keyless, run for example via Docker on port 8080.

What MCP tools does the server expose?

webshift_query, webshift_fetch and webshift_onboarding.

What license is it under?

The README states the MIT License.

Is Webshift free?

Yes. Webshift is open source, so you can use and self-host it for free.

What are the best alternatives to Webshift?

The closest alternatives to Webshift are searchio, free-web-mcp, webclaw, Open Agent Search and Ransack.

Where this comes from

  • Description: written from the project's README and its MCP Registry entry.

How AIPoweredBot checks servers