SkillFoxx

Claude Video Vision MCP

claude-video-vision

An MCP server and Claude Code plugin that let the agent watch video: frames via ffmpeg and audio transcription via Gemini, local Whisper or OpenAI

  • 1,500 ★ on the repository
  • Last commit
  • MIT
MCP server

Extended access

Why this level

  • Runs ffmpeg and yt-dlp and downloads Whisper models
  • The cloud backends send audio to Gemini or OpenAI
All reasons and checks

jordanrendric/claude-video-visionAuthor:Jordan Vasconcelos

Install

In your terminal, with SkillFoxx CLI

npx skillfoxx add mcp/claude-video-vision

Detects the agents on your machine, shows the access level and pins the version.

Other ways to install

Access and checks

How we review

Extended accessThe tool runs code or makes network calls and only reads external data.

  • Runs ffmpeg and yt-dlp and downloads Whisper models
  • The cloud backends send audio to Gemini or OpenAI

This is third-party code. Review the repository files before installing.

What it does

The MCP server extracts video frames via ffmpeg and processes audio with one of three backends: Gemini API, local Whisper or OpenAI API. The agent receives frames as images and audio transcription with timestamps; the server is a perception layer, not an interpretation layer. Local files and YouTube URLs are supported, downloaded with yt-dlp and preferring captions. It exposes six tools: video_info, video_analyze, video_watch, video_detail, video_configure, video_setup. The plugin adds a /watch-video slash command, a setup wizard and a video-perception skill that guides the agent's workflow. Frame options and the Whisper model are configurable.

Who it is for. For developers and analysts who need the agent to understand video content.

Good fit when

  • You want the agent to watch a local video or a YouTube clip and answer about its content
  • You need audio transcription with timestamps alongside frames
  • You need to analyze a bug report or a screen recording frame by frame

Not a fit when

  • You only need text analysis without video
  • You cannot install ffmpeg, or yt-dlp for YouTube URLs

Example request

Watch this video and tell me what happens in the first minute: ~/Downloads/demo.mp4

MCP

Transport
stdio
Authentication
not required

Environment variables

GEMINI_API_KEY secret
Gemini API key for the Gemini backend, has a free tier.
OPENAI_API_KEY secret
OpenAI key for the OpenAI backend, paid per usage.

Limitations

Needs Node.js 20 or newer and ffmpeg, plus yt-dlp for YouTube URLs. The Gemini API backend needs a GEMINI_API_KEY, the OpenAI backend needs an OPENAI_API_KEY and paid per-usage billing, and both services are unavailable from Russia without a VPN. The local whisper.cpp backend runs offline and keyless, with Whisper models downloaded on first use. At the first release it was tested on macOS with the local backend.

How to disable. Remove the claude-video-vision plugin via /plugin or drop the server from the client's MCP configuration.

FAQ

Can it work without cloud keys?

Yes. The local whisper.cpp backend transcribes audio offline, with no keys and no cost. Keys are needed only by the Gemini and OpenAI backends.

Is YouTube supported?

Yes. You can pass a URL directly; the server downloads the clip with yt-dlp and prefers captions, using the audio backend only when captions fall short.

How is Claude Video Vision different from transcribing video by hand?

The MCP server extracts frames through ffmpeg and processes audio with one of three backends: Gemini API, local Whisper or OpenAI API. The agent gets frames as images and an audio transcript with timestamps.

Can I use Claude Video Vision without cloud keys?

Yes, if you pick local Whisper for audio. Gemini and OpenAI need API keys. Install in Claude Code with /plugin marketplace add and the repository address from the install block. It also works in Cursor and Claude Desktop.

Discussion

No questions or reviews yet.

MCP servers

MCP server, /video plugin and CLI: transcript, key frames, on-screen text and metadata for a YouTube, Loom or local video

MCP serverExtended access122
Official

An agent in Cursor or Claude Desktop describes video, splits it into chapters and shots and finds moments by speech or picture

MCP serverExtended access14

ClipForge MCP

ClipForge

Official

ClipForge MCP builds a vertical video from a topic or a product link: stock footage, Edge TTS voiceover and captions, no keys, composed with FFmpeg

MCP serverExtended access1.3K

watch-cli is a skill and CLI that hands an agent the frames and transcript of any social video so it can read the clip itself.

SkillExtended access260

Claude Real Video

claude-real-video

A skill that lets Claude watch a video: it pulls scene-aware, deduplicated frames and a speech transcript through whisper from a link or file

SkillExtended access2.3K

A local MCP server sends images, video, audio and music to a multimodal model and returns a description to a text-only agent.

MCP serverExtended access120

README in short

The README describes a Claude Code plugin with an MCP server that gives the agent video perception. The server extracts frames via ffmpeg and transcribes audio via the Gemini API, local Whisper or OpenAI API, returning frames and text with timestamps. Local files and YouTube URLs via yt-dlp are supported. The plugin installs with /plugin commands, the server is pulled via npx, and configuration runs through the /setup-video-vision wizard. There are six MCP tools and a /watch-video slash command. MIT licensed.

README badge

Report a problem