Skip to content
MCP server

Local AI MCP

Local MCP server to check VRAM fit, manage models and delegate work to local LLM runtimes

Runs locallyStatus and history
Install
Pricing
Open source
Platforms
CLI, macOS, Linux
Runs
Locally
Version
0.6.1
Listing updated
3 days ago

About Local AI MCP

Local AI MCP is an open-source MCP server for ai models and llm tooling. It runs locally on your machine and is started by your AI client.

Local AI MCP is a control plane for models on your own machine, covering Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and any OpenAI-compatible server. It runs over stdio only and never opens a network listener, and a call without a provider argument only uses live local providers that have the model.

Its distinguishing piece is fit_check, which sizes the KV cache from the model's real layer architecture, sums VRAM across GPUs, models partial GPU and CPU offload and understands Apple Silicon unified memory. The delegate tool picks, readies and runs the best local model for a task in one call, and every result reports provider, host and whether it was local.

Hosted providers such as Moonshot are used only when named explicitly or when LOCAL_AI_ALLOW_HOSTED_FALLBACK is set. Install with npx -y @tmhs/local-ai-mcp, or use the .mcpb bundle for Claude Desktop. Defaults work for runtimes on their standard ports.

Install

Run in a terminal

claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcp

Add to claude_desktop_config.json

{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "local-ai-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "local-ai-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@tmhs/local-ai-mcp"
      ],
      "env": {
        "VLLM_API_KEY": "<VLLM_API_KEY>",
        "OPENAI_COMPAT_API_KEY": "<OPENAI_COMPAT_API_KEY>",
        "MOONSHOT_API_KEY": "<MOONSHOT_API_KEY>"
      }
    }
  }
}

Configuration

  • OLLAMA_HOSTOllama base URL
  • LMSTUDIO_HOSTLM Studio server base URL
  • LLAMACPP_HOSTllama.cpp server (llama-server) base URL
  • DOCKER_MODEL_RUNNER_HOSTDocker Model Runner host TCP endpoint
  • MLX_HOSTmlx_lm.server base URL (opt-in; Apple Silicon)
  • VLLM_HOSTvLLM OpenAI server base URL (opt-in)
  • VLLM_API_KEYBearer token for VLLM_HOST (when vLLM runs with --api-key)
  • FOUNDRY_LOCAL_HOSTFoundry Local service URL (opt-in; port from `foundry service status`)
  • OPENAI_COMPAT_HOSTAny OpenAI-compatible /v1 server (opt-in)
  • OPENAI_COMPAT_API_KEYBearer token for OPENAI_COMPAT_HOST
  • MOONSHOT_API_KEYMoonshot AI (Kimi) key. Hosted: only used when named explicitly.
  • LOCAL_AI_ALLOW_HOSTED_FALLBACKAllow implicit routing to hosted providers (default: off)
  • MOONSHOT_HOSTMoonshot API base URL
  • LOCAL_AI_DISABLEComma-separated provider names to skip probing
  • LOCAL_AI_PROVIDERSJSON array of extra provider instances: {type, host, name?, apiKey?, kind?}
  • LOCAL_AI_DISABLE_NETWORK_SEARCHKeep search_models offline (catalog only)
  • LOCAL_AI_BENCHMARK_CACHEBenchmark results file used to rank delegate by measured speed (default: per-user cache dir; off disables)

Tools this server provides (19)

Taken from the project documentation; the running server may differ.

delegate
Pick, ready and run the best local model for a task.
complete
Completion with JSON-schema output, images and streaming.
embed
Embeddings, one vector per input.
ensure_model
Pull, fit-check, evict-to-fit, load and verify.
fit_check
Whether a model fits on GPU, partial, CPU or not at all.
suggest_model
Best models for a task on this machine.
search_models
Hugging Face GGUF search with per-quant fit.
search_available
Offline curated catalog.
list_providers
Runtimes, kind, live status and capabilities.
list_models
Installed models per runtime.
list_loaded
Resident models and VRAM use.
model_info
Metadata and layer architecture.
pull_model
Download with progress.
remove_model
Remove a model after confirmation.
load_model
Load with keep-alive and context length.
unload_model
Free VRAM and RAM.
benchmark
Time to first token and tokens per second.
health_check
Reachability, version and runtime metrics.
system_resources
RAM, CPU, GPUs and VRAM.

What it does

Hardware-aware fit check
Reports whether a model will run on GPU, partially offloaded or CPU, with numbers.
One-call delegation
delegate chooses, loads and runs the best local model for a task.
Multi-runtime lifecycle
Pull, load, unload and remove models on the runtime that has them.
Local-first routing
Hosted APIs are opt-in; each result shows where inference ran.
Benchmarking
Measure time to first token and tokens per second across models.
Model search
Hugging Face GGUF search with per-quantization fit.

Good for

  • Check whether a 70B model fits a 12 GB card
  • Summarize private notes without sending them to the cloud
  • Free GPU memory by unloading idle models
  • Find the fastest installed coding model

Pricing

Open source

Free, runs locally

Prices change. Check tmhsdigital.github.io before you buy.

Local AI MCP FAQ

Does it send prompts to the cloud?

Not by default. Hosted providers are used only when named or when fallback is enabled.

Which runtimes are supported?

Ollama, LM Studio, llama.cpp, Docker Model Runner, MLX, vLLM, Foundry Local and OpenAI-compatible servers.

How do I install it?

Run npx -y @tmhs/local-ai-mcp, or open the .mcpb bundle in Claude Desktop.

Does it open a network port?

No. It runs over stdio only.

Is Local AI MCP free?

Yes. Local AI MCP is open source, so you can use and self-host it for free.

How do I add Local AI MCP to Claude Code?

Run this command in your terminal: claude mcp add local-ai-mcp --env VLLM_API_KEY='<VLLM_API_KEY>' --env OPENAI_COMPAT_API_KEY='<OPENAI_COMPAT_API_KEY>' --env MOONSHOT_API_KEY='<MOONSHOT_API_KEY>' -- npx -y @tmhs/local-ai-mcp

What are the best alternatives to Local AI MCP?

The closest alternatives to Local AI MCP are REA, Codebase Memory, GitHub, Serena MCP: the IDE for your agent and XERJ.

Alternatives to Local AI MCP

  1. REA

    MCP server

    CLI and MCP server that lets coding agents reverse engineer apps and binaries

    Runs locally
    • Open source
    • 464K GitHub stars
    • Developer tools
  2. Codebase Memory

    MCP server

    Local MCP server that builds a code knowledge graph for AI coding agents

    Runs locally
    • Open source
    • 48.7K GitHub stars
    • Knowledge and memory
  3. GitHub

    MCP server

    Official MCP server connecting AI assistants to GitHub repos, issues and workflows

    Not checked yet
    • Remote
    • Open source
    • 34.5K GitHub stars
    • Developer tools
  4. MCP toolkit giving coding agents symbol-level code retrieval and editing

    Runs locally
    • Open source
    • 31.5K GitHub stars
    • Developer tools
  5. XERJ

    MCP server

    Local search engine for AI agents with cited answers over your code and docs

    Runs locally
    • Open source
    • 19.2K GitHub stars
    • Knowledge and memory
  6. Ouroboros

    MCP server

    Spec-first layer that pins requirements and verifies results of AI coding agent work

    Runs locally
    • Open source
    • 6.4K GitHub stars
    • Developer tools