Local AI MCP
Local MCP server to check VRAM fit, manage models and delegate work to local LLM runtimes
- Pricing
- Open source
- Platforms
- CLI, macOS, Linux
- Runs
- Locally
- Version
- 0.6.1
- Listing updated
- 3 days ago
About Local AI MCP
Local AI MCP is an open-source MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.
Local AI MCP is a control plane for models on your own machine, covering Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and any OpenAI-compatible server. It runs over stdio only and never opens a network listener, and a call without a provider argument only uses live local providers that have the model.
Its distinguishing piece is fit_check, which sizes the KV cache from the model's real layer architecture, sums VRAM across GPUs, models partial GPU and CPU offload and understands Apple Silicon unified memory. The delegate tool picks, readies and runs the best local model for a task in one call, and every result reports provider, host and whether it was local.
Hosted providers such as Moonshot are used only when named explicitly or when LOCAL_AI_ALLOW_HOSTED_FALLBACK is set. Install with npx -y @tmhs/local-ai-mcp, or use the .mcpb bundle for Claude Desktop. Defaults work for runtimes on their standard ports.
Install
Run in a terminal
claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcpAdd to claude_desktop_config.json
{
"mcpServers": {
"local-ai-mcp": {
"command": "npx",
"args": [
"-y",
"@tmhs/local-ai-mcp"
],
"env": {
"VLLM_API_KEY": "<VLLM_API_KEY>",
"OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
"MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
}
}
}
}Add to ~/.cursor/mcp.json
{
"mcpServers": {
"local-ai-mcp": {
"command": "npx",
"args": [
"-y",
"@tmhs/local-ai-mcp"
],
"env": {
"VLLM_API_KEY": "<VLLM_API_KEY>",
"OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
"MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
}
}
}
}Add to .vscode/mcp.json
{
"servers": {
"local-ai-mcp": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@tmhs/local-ai-mcp"
],
"env": {
"VLLM_API_KEY": "<VLLM_API_KEY>",
"OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
"MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json
{
"mcpServers": {
"local-ai-mcp": {
"command": "npx",
"args": [
"-y",
"@tmhs/local-ai-mcp"
],
"env": {
"VLLM_API_KEY": "<VLLM_API_KEY>",
"OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
"MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
}
}
}
}Add to your client's MCP settings (most use this shape)
{
"mcpServers": {
"local-ai-mcp": {
"command": "npx",
"args": [
"-y",
"@tmhs/local-ai-mcp"
],
"env": {
"VLLM_API_KEY": "<VLLM_API_KEY>",
"OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
"MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
}
}
}
}Configuration
OLLAMA_HOSTOllama base URLLMSTUDIO_HOSTLM Studio server base URLLLAMACPP_HOSTllama.cpp server (llama-server) base URLDOCKER_MODEL_RUNNER_HOSTDocker Model Runner host TCP endpointMLX_HOSTmlx_lm.server base URL (opt-in; Apple Silicon)VLLM_HOSTvLLM OpenAI server base URL (opt-in)VLLM_API_KEYBearer token for VLLM_HOST (when vLLM runs with --api-key)FOUNDRY_LOCAL_HOSTFoundry Local service URL (opt-in; port from `foundry service status`)OPENAI_COMPAT_HOSTAny OpenAI-compatible /v1 server (opt-in)OPENAI_COMPAT_API_KEYBearer token for OPENAI_COMPAT_HOSTMOONSHOT_API_KEYMoonshot AI (Kimi) key. Hosted: only used when named explicitly.LOCAL_AI_ALLOW_HOSTED_FALLBACKAllow implicit routing to hosted providers (default: off)MOONSHOT_HOSTMoonshot API base URLLOCAL_AI_DISABLEComma-separated provider names to skip probingLOCAL_AI_PROVIDERSJSON array of extra provider instances: {type, host, name?, apiKey?, kind?}LOCAL_AI_DISABLE_NETWORK_SEARCHKeep search_models offline (catalog only)LOCAL_AI_BENCHMARK_CACHEBenchmark results file used to rank delegate by measured speed (default: per-user cache dir; off disables)
Tools this server provides (19)
Taken from the project documentation; the running server may differ.
- delegate
- Pick, ready and run the best local model for a task.
- complete
- Completion with JSON-schema output, images and streaming.
- embed
- Embeddings, one vector per input.
- ensure_model
- Pull, fit-check, evict-to-fit, load and verify.
- fit_check
- Whether a model fits on GPU, partial, CPU or not at all.
- suggest_model
- Best models for a task on this machine.
- search_models
- Hugging Face GGUF search with per-quant fit.
- search_available
- Offline curated catalog.
- list_providers
- Runtimes, kind, live status and capabilities.
- list_models
- Installed models per runtime.
- list_loaded
- Resident models and VRAM use.
- model_info
- Metadata and layer architecture.
- pull_model
- Download with progress.
- remove_model
- Remove a model after confirmation.
- load_model
- Load with keep-alive and context length.
- unload_model
- Free VRAM and RAM.
- benchmark
- Time to first token and tokens per second.
- health_check
- Reachability, version and runtime metrics.
- system_resources
- RAM, CPU, GPUs and VRAM.
What it does
- Hardware-aware fit check
- Reports whether a model will run on GPU, partially offloaded or CPU, with numbers.
- One-call delegation
- delegate chooses, loads and runs the best local model for a task.
- Multi-runtime lifecycle
- Pull, load, unload and remove models on the runtime that has them.
- Local-first routing
- Hosted APIs are opt-in; each result shows where inference ran.
- Benchmarking
- Measure time to first token and tokens per second across models.
- Model search
- Hugging Face GGUF search with per-quantization fit.
Good for
- Check whether a 70B model fits a 12 GB card
- Summarize private notes without sending them to the cloud
- Free GPU memory by unloading idle models
- Find the fastest installed coding model
Pricing
Open source
Free, runs locally
Prices change. Check tmhsdigital.github.io before you buy.
Local AI MCP FAQ
Does it send prompts to the cloud?
Not by default. Hosted providers are used only when named or when fallback is enabled.
Which runtimes are supported?
Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and OpenAI-compatible servers.
How do I install it?
Run npx -y @tmhs/local-ai-mcp, or open the .mcpb bundle in Claude Desktop.
Does it open a network port?
No. It runs over stdio only.
Is Local AI MCP free?
Yes. Local AI MCP is open source, so you can use and self-host it for free.
How do I add Local AI MCP to Claude Code?
Run this command in your terminal: claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcp
What are the best alternatives to Local AI MCP?
The closest alternatives to Local AI MCP are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.
Alternatives to Local AI MCP

REA
MCP serverCLI and MCP server that lets coding agents reverse engineer apps and binaries
Runs locally- Open source
- 464K GitHub stars
- Developer tools

Codebase Memory
MCP serverLocal MCP server that builds a code knowledge graph for AI coding agents
Runs locally- Open source
- 48.7K GitHub stars
- Knowledge and memory

GitHub
MCP serverOfficial MCP server connecting AI assistants to GitHub repos, issues and workflows
Not checked yet- Remote
- Open source
- 34.5K GitHub stars
- Developer tools

Serena MCP: the IDE for your agent
MCP serverMCP toolkit giving coding agents symbol-level code retrieval and editing
Runs locally- Open source
- 31.5K GitHub stars
- Developer tools

XERJ
MCP serverLocal search engine for AI agents with cited answers over your code and docs
Runs locally- Open source
- 19.2K GitHub stars
- Knowledge and memory

Ouroboros
MCP serverSpec-first layer that pins requirements and verifies results of AI coding agent work
Runs locally- Open source
- 6.4K GitHub stars
- Developer tools