# Local AI MCP

> Local AI MCP is an open-source MCP server for ai models and llm tooling. Local MCP server to check VRAM fit, manage models and delegate work to local LLM runtimes. It runs locally on your machine and is started by your AI client.

- **Kind:** MCP server
- **Pricing:** Open source
- **Platforms:** CLI, macOS, Linux, Windows, self-hosted
- **Website:** https://tmhsdigital.github.io/local-ai-mcp/
- **Repository:** https://github.com/tmhsdigital/local-ai-mcp
- **MCP Registry name:** io.github.TMHSDigital/local-ai-mcp
- **Categories:** AI models and LLM tooling, developer tools
- **Listing updated:** 2026-11-14

## About

Local AI MCP is a control plane for models on your own machine, covering Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and any OpenAI-compatible server. It runs over stdio only and never opens a network listener, and a call without a provider argument only uses live local providers that have the model.

Its distinguishing piece is fit_check, which sizes the KV cache from the model's real layer architecture, sums VRAM across GPUs, models partial GPU and CPU offload and understands Apple Silicon unified memory. The delegate tool picks, readies and runs the best local model for a task in one call, and every result reports provider, host and whether it was local.

Hosted providers such as Moonshot are used only when named explicitly or when LOCAL_AI_ALLOW_HOSTED_FALLBACK is set. Install with npx -y @tmhs/local-ai-mcp, or use the .mcpb bundle for Claude Desktop. Defaults work for runtimes on their standard ports.

## Features

- **Hardware-aware fit check:** Reports whether a model will run on GPU, partially offloaded or CPU, with numbers.
- **One-call delegation:** delegate chooses, loads and runs the best local model for a task.
- **Multi-runtime lifecycle:** Pull, load, unload and remove models on the runtime that has them.
- **Local-first routing:** Hosted APIs are opt-in; each result shows where inference ran.
- **Benchmarking:** Measure time to first token and tokens per second across models.
- **Model search:** Hugging Face GGUF search with per-quantization fit.

## Good for

- Check whether a 70B model fits a 12 GB card
- Summarize private notes without sending them to the cloud
- Free GPU memory by unloading idle models
- Find the fastest installed coding model

## Install

### Claude Code (Terminal)

```sh
claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcp
```

### Claude Desktop (claude_desktop_config.json)

```json
{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}
```

### Cursor (~/.cursor/mcp.json)

```json
{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}
```

### VS Code (.vscode/mcp.json)

```json
{
  "servers": {
    "local-ai-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}
```

### Windsurf (~/.codeium/windsurf/mcp_config.json)

```json
{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}
```

### Other clients (your client's MCP settings (most use this shape))

```json
{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}
```

## Tools

- `delegate`: Pick, ready and run the best local model for a task.
- `complete`: Completion with JSON-schema output, images and streaming.
- `embed`: Embeddings, one vector per input.
- `ensure_model`: Pull, fit-check, evict-to-fit, load and verify.
- `fit_check`: Whether a model fits on GPU, partial, CPU or not at all.
- `suggest_model`: Best models for a task on this machine.
- `search_models`: Hugging Face GGUF search with per-quant fit.
- `search_available`: Offline curated catalog.
- `list_providers`: Runtimes, kind, live status and capabilities.
- `list_models`: Installed models per runtime.
- `list_loaded`: Resident models and VRAM use.
- `model_info`: Metadata and layer architecture.
- `pull_model`: Download with progress.
- `remove_model`: Remove a model after confirmation.
- `load_model`: Load with keep-alive and context length.
- `unload_model`: Free VRAM and RAM.
- `benchmark`: Time to first token and tokens per second.
- `health_check`: Reachability, version and runtime metrics.
- `system_resources`: RAM, CPU, GPUs and VRAM.

## FAQ

### Does it send prompts to the cloud?

Not by default. Hosted providers are used only when named or when fallback is enabled.

### Which runtimes are supported?

Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and OpenAI-compatible servers.

### How do I install it?

Run npx -y @tmhs/local-ai-mcp, or open the .mcpb bundle in Claude Desktop.

### Does it open a network port?

No. It runs over stdio only.

### Is Local AI MCP free?

Yes. Local AI MCP is open source, so you can use and self-host it for free.

### How do I add Local AI MCP to Claude Code?

Run this command in your terminal: claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcp

### What are the best alternatives to Local AI MCP?

The closest alternatives to Local AI MCP are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.

## Alternatives

- [REA](https://ai-powered.com/mcp/rea): REA is an open-source MCP server for developer tools.
- [Codebase Memory](https://ai-powered.com/mcp/codebase-memory): Codebase Memory is an open-source MCP server for knowledge and memory.
- [GitHub](https://ai-powered.com/mcp/github-2): GitHub is an open-source MCP server for developer tools.
- [Serena MCP: the IDE for your agent](https://ai-powered.com/mcp/serena-mcp-the-ide-for-your-agent): Serena MCP: the IDE for your agent is an open-source MCP server for developer tools.
- [XERJ](https://ai-powered.com/mcp/xerj): XERJ is an open-source MCP server for knowledge and memory.
- [Ouroboros](https://ai-powered.com/mcp/ouroboros): Ouroboros is an open-source MCP server for developer tools.
- [Claude Code Ultimate Guide](https://ai-powered.com/mcp/claude-code-ultimate-guide): Claude Code Ultimate Guide is an open-source MCP server for ai models and llm tooling.
- [Concord MCP](https://ai-powered.com/mcp/concord-mcp): Concord MCP is a freemium MCP server for developer tools.

---
Source: https://ai-powered.com/mcp/local-ai-mcp
