/claude-real-video-for-agents

name: claude-real-video-for-agents
description: "Install and use crv (claude-real-video) — a tool that lets any AI agent watch videos by extracting scene-aware keyframes, deduplicating them, and transcribing audio. Use when the user shares a video URL or file and wants it analyzed, summarized, or discussed."

claude-real-video for AI agents

What is crv?

`crv` (claude-real-video) is a CLI tool that extracts meaningful frames and transcripts from videos so AI agents can "see" and "read" them. It uses scene-change detection (not fixed-interval sampling), sliding-window deduplication, and optional Whisper transcription.

**Key advantage**: Same 58-second clip at fixed 1fps = 58 frames. crv keeps the **26 that actually differ**, and `--grid` packs them into **3 contact sheets**. Fewer tokens, nothing missed.

Installation

Prerequisites

• Python 3.10+
• ffmpeg / ffprobe on PATH

# macOS
brew install ffmpeg

# Linux
sudo apt install ffmpeg

# Windows
winget install Gyan.FFmpeg

Install crv

# Recommended: with audio transcription support
pip install "claude-real-video[whisper]"

# Core only (frames + dedup)
pip install claude-real-video

The `[whisper]` extra never installs itself — without it there is **no speech-to-text** (videos that ship their own subtitles still get a transcript).

Verify installation

crv --help
ffmpeg -version

Install as agent skill

Run the bundled installer to symlink this skill into all detected agent platforms:

bash install-skill.sh

Or manually copy to your agent's skill directory:

# Claude Code
cp -r skills/claude-real-video-for-agents ~/.claude/skills/

# Codex
cp -r skills/claude-real-video-for-agents ~/.codex/skills/

# OpenCode
cp -r skills/claude-real-video-for-agents ~/.opencode/skills/

# Gemini CLI
cp -r skills/claude-real-video-for-agents ~/.gemini/skills/

Usage

Basic: Watch a video from URL

crv "https://www.youtube.com/watch?v=VIDEO_ID"

Output in `crv-out/`:

• `frames/` — deduplicated keyframes
• `transcript.txt` — plain-text transcript
• `MANIFEST.txt` — summary for LLM consumption

Recommended: With grid and intent

crv "https://youtu.be/VIDEO_ID" -o crv-out --grid --why "what the user wants to know"

• `--grid` — tiles frames into 3x3 contact sheets (cuts image count ~9x)
• `--why` — focuses the analysis on a specific question

Local file with transcript

crv lecture.mp4 -o out --lang en

Frames only (no transcription — much faster)

crv clip.mp4 --no-transcribe

Login-gated video

crv "https://..." --cookies cookies.txt
crv "https://..." --cookies-from-browser chrome

Slow-changing content (animations, tutorials)

crv tutorial.mp4 --adaptive

Save to knowledge base

crv "https://youtu.be/..." --why "pricing strategy" --kb ~/notes

View what the model will see

crv video.mp4 --viewer
# Opens viewer.html — video + keyframes + transcript, fully offline

Agent Workflow

When a user shares a video (URL or file path):

1. **Run crv** with `--grid` and `--why`:

   crv "<url-or-path>" -o crv-out --grid --why "<user's question>"

For long videos, cap frames: `--max-frames 60`

Use one output folder per video (e.g. `-o crv-out/<slug>`). A folder that already holds an analysis is refused; pass `--overwrite` to replace it.

2. **Read `MANIFEST.txt` first** — it summarizes the run (frame counts, frames dir) and includes the transcript. Frames are named in chronological order; per-segment transcript timings live in `transcript.json` when available (there are no per-frame timestamps).

3. **Read contact sheets** in `crv-out/grids/` (each is a 3x3 sequence of consecutive keyframes, chronological). Only read individual `crv-out/frames/*.jpg` when you need a close-up.

4. **Answer the user's question**, citing transcript timings (from `transcript.json`) where available.

CLI Reference

| Flag | Default | Description | |---|---|---| | `source` (positional) | — | Video URL or local file path | | `-o, --out` | `crv-out` | Output directory | | `--overwrite` | off | Replace a previous analysis living in the output directory (without this, a non-empty output dir is refused to avoid mixing videos) | | `--scene` | `0.30` | Scene-change sensitivity (0-1, lower = more frames) | | `--fps-floor` | `1.0` | Guarantee at least one frame every N seconds | | `--max-frames` | `150` | Hard cap on total frames | | `--adaptive` | off | Adaptive scene detection for slow-changing content | | `--text-anchors` | off | Force frames at subtitle-cue timestamps — needs a sidecar `.srt`/`.vtt` or embedded subtitle track (burned-in captions can't be detected) | | `--lang` | `auto` | Whisper language (`en`, `zh`, `auto`, etc.) | | `--cookies` | — | Netscape cookie file for login-gated sources | | `--cookies-from-browser` | — | Read cookies from browser (`chrome`, `safari`, `firefox`, `edge`) | | `--no-transcribe` | off | Skip audio transcription | | `--viewer` | off | Write a local `viewer.html` | | `--whisper-model` | `base` | Whisper model size (`tiny`, `base`, `small`, `medium`, `large`, `turbo` — turbo: near large-v2 accuracy, ~8x faster) | | `--dedup-threshold` | `8` | % of pixels that must change for a new frame (higher = fewer frames kept) | | `--dedup-window` | `4` | Compare against last N kept frames (1 = consecutive-only) | | `--report` | off | Keep dropped frames + write `report.html` | | `--why` | — | Viewing intent, e.g. `--why "find the pricing strategy"` — focuses the model's analysis | | `--grid` | off | Tile frames into 3x3 contact sheets | | `--kb` | — | Save as dated markdown note to knowledge-base folder | | `--keep-audio` | off | Save full soundtrack as `audio.m4a` (for Gemini, GPT-4o, etc.) |

Python API

from claude_real_video import process

result = process("https://youtu.be/...", "out", lang="en")
pri