/watch-video

name: watch-video
description: When you want to extract content from a video — YouTube, Loom, Vimeo, Riverside, Zoom recording, local MP4, X/IG video, anything yt-dlp supports. Three depth modes user picks per invocation — transcript (just words, fast/free), visual (transcript + ffmpeg frame extraction + Claude vision pass on key moments), multimodal (Gemini native video ingestion if $GEMINI_API_KEY set, else dense Claude vision). Uses local Whisper for transcription (MLX-Whisper on Apple Silicon, faster-whisper elsewhere), falls back to platform-provided transcripts when available (Loom, Riverside, YouTube auto-subs). Saves to ~/Documents/videos/<source>-<slug>-<date>/ and optionally captures summary to second-brain raw/ as call-/meeting-/note-. Triggers on "/watch-video <url>," "watch this video," "transcribe this loom," "analyze this video," "summarize this recording," "key moments from this," "what happened in this video." This skill replaces and broadens the prior youtube-transcript skill.
metadata:
  version: 0.2.3

/watch-video — Transcribe and analyze any video at the depth you choose

Replaces and broadens the prior `youtube-transcript` skill. YouTube is now one of many sources; depth is user-controlled.

Step 1 — Parse input

Accept:

• **YouTube**: full URL, `youtu.be/<id>`, `youtube.com/shorts/<id>`, raw 11-char ID
• **Loom**: `loom.com/share/<id>` or `loom.com/embed/<id>`
• **Vimeo**: `vimeo.com/<id>`
• **Riverside**: download URL or local file
• **Zoom**: local `.mp4` from a downloaded recording
• **X / IG / TikTok video**: URL — defers to `social-fetch` for metadata, uses yt-dlp for the file
• **Local file**: any path to an `.mp4` / `.mov` / `.webm` / `.mkv`

Detect source from URL pattern or file extension. If ambiguous, ask.

Step 2 — Parse depth mode

| Invocation | Mode | What you get | |---|---|---| | `/watch-video <url>` | **transcript** (default) | Clean text, metadata, optional chapters | | `/watch-video <url> transcript` | transcript | Same as default | | `/watch-video <url> visual` | visual | Transcript + frames at intervals + Claude vision pass identifying key moments | | `/watch-video <url> multimodal` | multimodal | Native video to Gemini (if `$GEMINI_API_KEY`), else dense Claude vision frame-by-frame |

If the depth isn't specified and the video is >10 minutes, ask before defaulting (visual/multimodal cost real money on long videos).

Step 3 — Pull metadata

For URL sources, use yt-dlp:

yt-dlp --print "%(title)s|%(uploader)s|%(duration_string)s|%(upload_date>%Y-%m-%d)s|%(description)s" \
  --print "%(chapters)j" --skip-download "<url>"

Capture: title, uploader/channel, duration, upload date, description (first paragraph), chapters (JSON or null).

For local files, use ffprobe:

ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "<file>"

Step 4 — Build workdir

~/Documents/videos/<source>-<slug>-<date>/

Where:

• `source`: `youtube` / `loom` / `vimeo` / `riverside` / `zoom` / `local`
• `slug`: kebab-case of title (first 4–6 words, max 50 chars)
• `date`: `YYYY-MM-DD`

Step 5 — Get the transcript

**Backend selection** (in order):

1. **Platform-provided transcript** if it exists and looks complete:

• YouTube: `yt-dlp --write-sub --write-auto-sub --skip-download --sub-lang en --sub-format vtt`
• Loom: fetch via `https://www.loom.com/share/<id>` page metadata or Loom API if `$LOOM_API_KEY` set
• Riverside: built-in transcripts available on the recording's share page
• If platform transcript exists and has timestamps, use it. Skip Whisper.

2. **MLX-Whisper local** (default on Apple Silicon Macs):

   # Install once: pip install mlx-whisper
   python3 -c "import mlx_whisper; mlx_whisper.transcribe('<file>', path_or_hf_repo='mlx-community/whisper-large-v3-turbo')" \
     > "<workdir>/transcript-raw.json"

Or via the CLI: `mlx_whisper <file> --model mlx-community/whisper-large-v3-turbo --output-dir <workdir>`

3. **Not on Apple Silicon** (Linux, Intel Mac, Windows, cloud agents): `faster-whisper` (`pip install faster-whisper`, CPU or CUDA) or `whisper.cpp`. Same model size, same output handling.

Download the video file first if it's a URL (use yt-dlp; Loom/Vimeo/YT all supported):

yt-dlp -f "bv*[height<=720]+ba/b[height<=720]" -o "<workdir>/video.%(ext)s" "<url>"

720p is plenty for transcription and frame analysis (smaller download, faster processing).

**Clean the transcript** (only needed for YouTube auto-subs which have rolling captions; Whisper output is already clean):

# YouTube VTT cleanup — de-dup rolling captions, strip tags, paragraph-break on cue gaps >2s
awk '
  /^WEBVTT/ || /^Kind:/ || /^Language:/ || /^NOTE/ { next }
  /-->/ { in_cue = 1; last = ""; next }
  /^$/ { if (last) print last; in_cue = 0; last = ""; next }
  in_cue { gsub(/<[^>]+>/, "", $0); last = $0 }
  END { if (last) print last }
' "<workdir>/transcript.en.vtt" | awk '!seen[$0]++' > "<workdir>/transcript.txt"

Save final to `<workdir>/transcript.txt`.

Step 6 — If `transcript` mode: stop here

Output:

• `transcript.txt`
• `metadata.json`
• One-line summary in chat: title, source, duration, word count
• Path to workdir
• (Optional) Step 9 — offer to capture to second-brain

Step 7 — If `visual` mode: extract frames + vision pass

Frame extraction (ffmpeg)

Cadence by source heuristic:

| Source type | Frame cadence | |---|---| | Screen-share / Loom / demo | 1 frame per **5s** (UI changes fast) | | Talking head / podcast | 1 frame per **30s** (slow change) | | Slide presentation | 1 frame per **10s** + force a frame on each detected scene change | | Default if unsure | 1 frame per **15s** |

mkdir -p "<workdir>/frames"
ffmpeg -i "<workdir>/video.mp4" -vf "fps=1/15" "<workdir>/frames/frame-%04d.png" -y

For scene-change detection (slide decks especially):

ffmpeg -i "<workdir>/video.mp4" -vf "select='gt(scene,0.