Skip to content
MCP server

Web Search API Eval

Query a 30-question benchmark of You.com, Tavily, Exa and Parallel search APIs

Install
Pricing
Free
Platforms
CLI, API, Web
Runs
Locally or Remotely
Version
1.1.0
Listing updated
2 days ago

About Web Search API Eval

Web Search API Eval is a free MCP server for web search and scraping. It can run locally on your machine or as a hosted remote server.

This server publishes the results of a web search API benchmark so assistants can answer questions about it. The study compared You.com, Tavily, Exa and Parallel on 30 agent-style queries, 10 each for GTM, insurance and finance agents, run in promptfoo with three runs per query and client-side timing. Answers were graded with the published SimpleQA rubric and FreshQA strict rules, and the data is dated 8 October 2026.

The study's own conclusions are that You.com was fastest and most consistently fresh, Parallel led accuracy, Exa had the best precision@5 in GTM and insurance, and Tavily was strongest on GTM accuracy and docs lookups. The benchmark tools are read-only, and you can pull the scorecard, compare providers, ask for a recommendation for a given agent need, read any single question with each API's answer and score, view charts and read the methodology.

The local npm version adds run_live_search, which sends one query to each provider with your own API keys and credits. It is a spot check rather than a graded run. A hosted endpoint without any install is also available.

Install

Run in a terminal

claude mcp add --transport http websearch-api-eval https://websearch-api-eval.psberkeleymba.workers.dev/mcp

Add to Settings → Connectors → Add custom connector

https://websearch-api-eval.psberkeleymba.workers.dev/mcp
Install in CursorOpens Cursor, which asks you to confirm. Or add it by hand:

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "websearch-api-eval": {
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}
Install in VS CodeOpens VS Code, which asks you to confirm. Or add it by hand:

Add to .vscode/mcp.json

{
  "servers": {
    "websearch-api-eval": {
      "type": "http",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "websearch-api-eval": {
      "serverUrl": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "websearch-api-eval": {
      "type": "http",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Configuration

  • YDC_API_KEYOptional. You.com API key, used only by run_live_search
  • TAVILY_API_KEYOptional. Tavily API key, used only by run_live_search
  • EXA_API_KEYOptional. Exa API key, used only by run_live_search
  • PARALLEL_API_KEYOptional. Parallel API key, used only by run_live_search

Tools this server provides (12)

Taken from the project documentation; the running server may differ.

get_overall_results
Headline, overall scorecard, per-use-case summary and which API fits which agent.
get_scorecard
Full scorecard for overall, gtm, insurance or finance.
compare_providers
Two or more providers side by side.
recommend_provider
Which API fits an agent's need, with numbers.
list_questions
The 30 questions, filterable by use case.
get_question
One question with answer key, sources and each API's result.
get_charts
The published graphs, optionally an image.
get_methodology
How tests ran, the 10 metrics and limits.
how_to_run_locally
Steps and keys to rerun the full benchmark.
search
Generic search used by ChatGPT research features.
fetch
Generic document fetch.
run_live_search
Sends one query to each provider with your keys (local only).

What it does

Scorecards
Overall and per-use-case scores for GTM, insurance and finance.
Provider comparison
Side-by-side comparison of two or more providers, filterable by metric.
Question drill-down
See the task, search call, answer key, sources and each API's result.
Recommendation
Suggests which API fits an agent's need with supporting numbers.
Live spot check
Local mode sends one query to each provider using your own keys.
Hosted or local
Use the hosted URL with no install or run the npm package.

Good for

  • Pick a search API for a voice agent based on latency results
  • Compare Exa and Parallel on finance questions
  • Read what each API returned for one benchmark question
  • Rerun a quick live query across providers with your keys

Pricing

Free

Free; provider API keys optional and billed by those providers

Prices change. Check psinglap.github.io before you buy.

Web Search API Eval FAQ

Which providers were tested?

You.com, Tavily, Exa and Parallel.

How many queries were used?

30 queries: 10 each for GTM, insurance and finance agents.

Do I need API keys?

Only for run_live_search in the local version; the benchmark tools need none.

Is run_live_search available on the hosted server?

No, it is local only.

When were results collected?

The README gives the run date as 8 October 2026.

Is Web Search API Eval free?

Yes. Web Search API Eval is free to use.

How do I add Web Search API Eval to Claude Code?

Run this command in your terminal: claude mcp add --transport http websearch-api-eval https://websearch-api-eval.psberkeleymba.workers.dev/mcp

What are the best alternatives to Web Search API Eval?

The closest alternatives to Web Search API Eval are Scrapling MCP Server, Firecrawl MCP Server, Agent-Native Analytics, Google Ads + Meta Ads + SEO/GEO MCP and SearXNG Search.

Alternatives to Web Search API Eval

  1. MCP server for AI clients to scrape sites with HTTP and stealth browser sessions

    Runs locally
    • 17 tools
    • Open source
    • 104K GitHub stars
    • Web search and scraping
  2. MCP server that lets AI agents search, scrape and interact with web pages

    • 3 tools
    • Remote
    • Freemium
    • 7.8K GitHub stars
    • Web search and scraping
  3. Open-source analytics app where an AI agent builds charts and dashboards from your data

    • Remote
    • Open source
    • 7.7K GitHub stars
    • Analytics and data
  4. Hosted MCP connector for managing Google Ads, Meta Ads, GA4 and Search Console from chat

    • 8 tools
    • Remote
    • Freemium
    • 8.1K GitHub stars
    • Marketing and SEO
  5. SearXNG Search

    MCP server

    MCP server for web search and page reading through a SearXNG instance

    Runs locally
    • 3 tools
    • Open source
    • 1.4K GitHub stars
    • Web search and scraping
  6. Indian NSE and BSE stock market data for AI assistants through 48 MCP tools

    • 17 tools
    • Remote
    • Freemium
    • 150 GitHub stars
    • Finance and payments