Skip to content
MCP server

Video Understanding MCP

Ask questions about a video file or TikTok, YouTube or X link via a video-capable model

Runs locallyStatus and history
Install
Platforms
CLI, macOS, Linux
Runs
Locally
Version
1.0.2
Listing updated
today

Similar MCP servers

About Video Understanding MCP

Video Understanding MCP is an MCP server for design and media. It runs locally on your machine and is started by your AI client.

You supply a page URL, a direct media URL or an allowed local file plus a question; the server downloads and shrinks the video with ffmpeg, sends it to an OpenAI-compatible chat endpoint whose model accepts video, and returns a structured answer. The result holds an answer, a summary, speech with language and transcript, on-screen text, a timeline of moments and analysis metadata including usage and warnings.

Endpoints disagree on how a video is attached, so VIDEO_PART_FORMAT selects video_url, image_url (default), file or input_file. To avoid answers invented from nothing, it sends a parallel prompt without the video and compares prompt-token usage; if the endpoint evidently ignored the video, the call fails with MEDIA_NOT_PROCESSED. That doubles requests and can be switched off.

Local files are refused unless VIDEO_ALLOWED_DIRS is set. It needs Node.js 22.21 or newer and ffmpeg, plus yt-dlp for TikTok or YouTube page links, and the default analysed range is 300 seconds.

Install

Run in a terminal

claude mcp add video-understanding-mcp --env VIDEO_BASE_URL='<VIDEO_BASE_URL>' --env VIDEO_MODEL='<VIDEO_MODEL>' --env VIDEO_API_KEY='<VIDEO_API_KEY>' --env VIDEO_EXTRA_HEADERS='<VIDEO_EXTRA_HEADERS>' -- npx -y @granitebps/video-understanding-mcp

Add to claude_desktop_config.json

{
  "mcpServers": {
    "video-understanding-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@granitebps/video-understanding-mcp"
      ],
      "env": {
        "VIDEO_BASE_URL": "<VIDEO_BASE_URL>",
        "VIDEO_MODEL": "<VIDEO_MODEL>",
        "VIDEO_API_KEY": "<VIDEO_API_KEY>",
        "VIDEO_EXTRA_HEADERS": "<VIDEO_EXTRA_HEADERS>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "video-understanding-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@granitebps/video-understanding-mcp"
      ],
      "env": {
        "VIDEO_BASE_URL": "<VIDEO_BASE_URL>",
        "VIDEO_MODEL": "<VIDEO_MODEL>",
        "VIDEO_API_KEY": "<VIDEO_API_KEY>",
        "VIDEO_EXTRA_HEADERS": "<VIDEO_EXTRA_HEADERS>"
      }
    }
  }
}

Add to .vscode/mcp.json

{
  "servers": {
    "video-understanding-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@granitebps/video-understanding-mcp"
      ],
      "env": {
        "VIDEO_BASE_URL": "<VIDEO_BASE_URL>",
        "VIDEO_MODEL": "<VIDEO_MODEL>",
        "VIDEO_API_KEY": "<VIDEO_API_KEY>",
        "VIDEO_EXTRA_HEADERS": "<VIDEO_EXTRA_HEADERS>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "video-understanding-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@granitebps/video-understanding-mcp"
      ],
      "env": {
        "VIDEO_BASE_URL": "<VIDEO_BASE_URL>",
        "VIDEO_MODEL": "<VIDEO_MODEL>",
        "VIDEO_API_KEY": "<VIDEO_API_KEY>",
        "VIDEO_EXTRA_HEADERS": "<VIDEO_EXTRA_HEADERS>"
      }
    }
  }
}

Add to your client's MCP settings (most use this shape)

{
  "mcpServers": {
    "video-understanding-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@granitebps/video-understanding-mcp"
      ],
      "env": {
        "VIDEO_BASE_URL": "<VIDEO_BASE_URL>",
        "VIDEO_MODEL": "<VIDEO_MODEL>",
        "VIDEO_API_KEY": "<VIDEO_API_KEY>",
        "VIDEO_EXTRA_HEADERS": "<VIDEO_EXTRA_HEADERS>"
      }
    }
  }
}

Using several servers? Build one config file with the MCP config generator.

Configuration

  • VIDEO_BASE_URLBase URL of an OpenAI-compatible chat completions endpoint whose model accepts video, e.g. https://host/v1Required
  • VIDEO_MODELModel (or router alias) to send videos toRequired
  • VIDEO_API_KEYEndpoint API key, sent as a bearer token
  • VIDEO_EXTRA_HEADERSExtra endpoint request headers as a JSON object, such as Cloudflare Access service-token headers
  • VIDEO_PART_FORMATHow the video is attached: video_url, image_url, file, or input_file
  • VIDEO_MAX_DURATION_SECONDSLongest analyzed range in seconds
  • VIDEO_MAX_UPLOAD_MBLargest prepared video sent to the endpoint, in MB
  • VIDEO_ALLOWED_DIRSDirectories local files may be read from; local files are refused when unset
  • VIDEO_VERIFY_MEDIAFail calls when the endpoint evidently ignored the video

Tools this server provides (2)

Taken from the project documentation; the running server may differ.

understand_video
Analyses a video from a URL or local path and answers a question about it.
get_server_info
Shows version, endpoint host, model, limits and tool availability.

What it does

Structured answers
Returns answer, summary, speech transcript, on-screen text and a timeline.
Many sources
Accepts page URLs, direct media URLs with headers, or allowed local files.
Endpoint formats
Four request formats cover different OpenAI-compatible gateways.
Ignored-video check
Fails with MEDIA_NOT_PROCESSED if the endpoint did not use the video.
Range and limits
Analyse a start and end range within configurable duration and upload limits.

Good for

  • Summarise a TikTok or YouTube video
  • Pull on-screen text and timestamps from a clip
  • Transcribe speech in a short video
  • Review a local screen recording against a question
  • Analyse a short range of a workout video

Video Understanding MCP FAQ

What does it need?

Node.js 22.21.0 or newer, ffmpeg with ffprobe, an OpenAI-compatible endpoint with a video-capable model, and yt-dlp for page URLs.

Can it read local files?

Only inside directories listed in VIDEO_ALLOWED_DIRS; local files are refused when it is unset.

What is the default maximum length?

300 seconds of analysed range, configurable up to 7200.

How do I check setup?

Call the get_server_info tool to see versions, limits and whether ffmpeg and yt-dlp are found.

How do I add Video Understanding MCP to Claude Code?

Run this command in your terminal: claude mcp add video-understanding-mcp --env VIDEO_BASE_URL='<VIDEO_BASE_URL>' --env VIDEO_MODEL='<VIDEO_MODEL>' --env VIDEO_API_KEY='<VIDEO_API_KEY>' --env VIDEO_EXTRA_HEADERS='<VIDEO_EXTRA_HEADERS>' -- npx -y @granitebps/video-understanding-mcp

What are the best alternatives to Video Understanding MCP?

The closest alternatives to Video Understanding MCP are HyperFrames by HeyGen, headroom, omniget, next-ai-draw-io and mcp-for-blender.

Where this comes from

  • Description: written from the project's README.
  • Install steps: from the official MCP Registry entry.

How AIPoweredBot checks servers