MCP server
EvalForge Lite
Compare text LLMs across OpenRouter, Bedrock, Vertex AI and Foundry with automated…
Runs locallyStatus and history
- Pricing
- Open source
- Runs
- Locally
- Version
- 1.0.1
- Listing updated
- 5 days ago
About EvalForge Lite
EvalForge Lite is an open-source MCP server. It runs locally on your machine and is started by your AI client.
Compare text LLMs across OpenRouter, Bedrock, Vertex AI and Foundry with automated grading.
Install
Run in a terminal
claude mcp add evalforge-lite --env OPENROUTER_API_KEY='<OPENROUTER_API_KEY>' -- uvx evalforge-liteAdd to claude_desktop_config.json
{
"mcpServers": {
"evalforge-lite": {
"command": "uvx",
"args": [
"evalforge-lite"
],
"env": {
"OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
}
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"evalforge-lite": {
"command": "uvx",
"args": [
"evalforge-lite"
],
"env": {
"OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
}
}
}
}Add to .vscode/mcp.json
{
"servers": {
"evalforge-lite": {
"type": "stdio",
"command": "uvx",
"args": [
"evalforge-lite"
],
"env": {
"OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"evalforge-lite": {
"command": "uvx",
"args": [
"evalforge-lite"
],
"env": {
"OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
}
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"evalforge-lite": {
"command": "uvx",
"args": [
"evalforge-lite"
],
"env": {
"OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
}
}
}
}Configuration
OPENROUTER_API_KEYOptional. OpenRouter API key held by the server so tool calls need no credentials for OpenRouter. If unset, pass creds in each tool call.
Pricing
Open source
EvalForge Lite FAQ
Is EvalForge Lite free?
Yes. EvalForge Lite is open source, so you can use and self-host it for free.
How do I add EvalForge Lite to Claude Code?
Run this command in your terminal: claude mcp add evalforge-lite --env OPENROUTER_API_KEY='<OPENROUTER_API_KEY>' -- uvx evalforge-lite