
Web Search API Eval
Query a 30-question benchmark of You.com, Tavily, Exa and Parallel search APIs
- Pricing
- Free
- Platforms
- CLI, API, Web
- Runs
- Locally or Remotely
- Version
- 1.1.0
- Listing updated
- 2 days ago
About Web Search API Eval
Web Search API Eval is a free MCP server for web search and scraping. It can run locally on your machine or as a hosted remote server.
This server publishes the results of a web search API benchmark so assistants can answer questions about it. The study compared You.com, Tavily, Exa and Parallel on 30 agent-style queries, 10 each for GTM, insurance and finance agents, run in promptfoo with three runs per query and client-side timing. Answers were graded with the published SimpleQA rubric and FreshQA strict rules, and the data is dated 8 October 2026.
The study's own conclusions are that You.com was fastest and most consistently fresh, Parallel led accuracy, Exa had the best precision@5 in GTM and insurance, and Tavily was strongest on GTM accuracy and docs lookups. The benchmark tools are read-only, and you can pull the scorecard, compare providers, ask for a recommendation for a given agent need, read any single question with each API's answer and score, view charts and read the methodology.
The local npm version adds run_live_search, which sends one query to each provider with your own API keys and credits. It is a spot check rather than a graded run. A hosted endpoint without any install is also available.
Install
Run in a terminal
claude mcp add --transport http websearch-api-eval https://websearch-api-eval.psberkeleymba.workers.dev/mcpAdd to Settings → Connectors → Add custom connector
https://websearch-api-eval.psberkeleymba.workers.dev/mcpAdd to ~/.cursor/mcp.json
{
"mcpServers": {
"websearch-api-eval": {
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}Add to .vscode/mcp.json
{
"servers": {
"websearch-api-eval": {
"type": "http",
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"websearch-api-eval": {
"serverUrl": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"websearch-api-eval": {
"type": "http",
"url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
}
}
}Configuration
YDC_API_KEYOptional. You.com API key, used only by run_live_searchTAVILY_API_KEYOptional. Tavily API key, used only by run_live_searchEXA_API_KEYOptional. Exa API key, used only by run_live_searchPARALLEL_API_KEYOptional. Parallel API key, used only by run_live_search
Tools this server provides (12)
Taken from the project documentation; the running server may differ.
- get_overall_results
- Headline, overall scorecard, per-use-case summary and which API fits which agent.
- get_scorecard
- Full scorecard for overall, gtm, insurance or finance.
- compare_providers
- Two or more providers side by side.
- recommend_provider
- Which API fits an agent's need, with numbers.
- list_questions
- The 30 questions, filterable by use case.
- get_question
- One question with answer key, sources and each API's result.
- get_charts
- The published graphs, optionally an image.
- get_methodology
- How tests ran, the 10 metrics and limits.
- how_to_run_locally
- Steps and keys to rerun the full benchmark.
- search
- Generic search used by ChatGPT research features.
- fetch
- Generic document fetch.
- run_live_search
- Sends one query to each provider with your keys (local only).
What it does
- Scorecards
- Overall and per-use-case scores for GTM, insurance and finance.
- Provider comparison
- Side-by-side comparison of two or more providers, filterable by metric.
- Question drill-down
- See the task, search call, answer key, sources and each API's result.
- Recommendation
- Suggests which API fits an agent's need with supporting numbers.
- Live spot check
- Local mode sends one query to each provider using your own keys.
- Hosted or local
- Use the hosted URL with no install or run the npm package.
Good for
- Pick a search API for a voice agent based on latency results
- Compare Exa and Parallel on finance questions
- Read what each API returned for one benchmark question
- Rerun a quick live query across providers with your keys
Pricing
Free
Free; provider API keys optional and billed by those providers
Prices change. Check psinglap.github.io before you buy.
Web Search API Eval FAQ
Which providers were tested?
You.com, Tavily, Exa and Parallel.
How many queries were used?
30 queries: 10 each for GTM, insurance and finance agents.
Do I need API keys?
Only for run_live_search in the local version; the benchmark tools need none.
Is run_live_search available on the hosted server?
No, it is local only.
When were results collected?
The README gives the run date as 8 October 2026.
Is Web Search API Eval free?
Yes. Web Search API Eval is free to use.
How do I add Web Search API Eval to Claude Code?
Run this command in your terminal: claude mcp add --transport http websearch-api-eval https://websearch-api-eval.psberkeleymba.workers.dev/mcp
What are the best alternatives to Web Search API Eval?
The closest alternatives to Web Search API Eval are Scrapling MCP Server, Firecrawl MCP Server, Agent-Native Analytics, Google Ads + Meta Ads + SEO/GEO MCP and SearXNG Search.
Alternatives to Web Search API Eval

Scrapling MCP Server
MCP serverMCP server for AI clients to scrape sites with HTTP and stealth browser sessions
Runs locally- 17 tools
- Open source
- 104K GitHub stars
- Web search and scraping

Firecrawl MCP Server
MCP serverMCP server that lets AI agents search, scrape and interact with web pages
- 3 tools
- Remote
- Freemium
- 7.8K GitHub stars
- Web search and scraping

Agent-Native Analytics
MCP serverOpen-source analytics app where an AI agent builds charts and dashboards from your data
- Remote
- Open source
- 7.7K GitHub stars
- Analytics and data

Google Ads + Meta Ads + SEO/GEO MCP
MCP serverHosted MCP connector for managing Google Ads, Meta Ads, GA4 and Search Console from chat
- 8 tools
- Remote
- Freemium
- 8.1K GitHub stars
- Marketing and SEO

SearXNG Search
MCP serverMCP server for web search and page reading through a SearXNG instance
Runs locally- 3 tools
- Open source
- 1.4K GitHub stars
- Web search and scraping

Tapetide - Indian Stock Market MCP
MCP serverIndian NSE and BSE stock market data for AI assistants through 48 MCP tools
- 17 tools
- Remote
- Freemium
- 150 GitHub stars
- Finance and payments