/youtube-notetaker

name: youtube-notetaker
description: >
  Turn any YouTube talk into a studyable, interactive deep-dive stored as plain markdown:
  extract slide images at their timestamps, build a clean [HH:MM:SS] transcript, write editable
  notes, and save everything as one markdown file per video in a local library folder. A
  bundled zero-dependency server (serve.py) renders the whole library as a single-page artifact
  (front-page index + per-video split pane: slide deck + embedded player + searchable transcript)
  with notes that save back to the markdown files. Fully self-contained: no external services,
  configurable library path. Use when the user gives a YouTube URL and
  wants to study a talk, capture slides, take timestamped notes, or build a talk library.
  Triggers on: "deep-dive this talk", "extract slides from this video", "add this YouTube video
  to my study library", "take notes on this talk", followed by a YouTube URL.

YouTube Notetaker

Build a personal library of YouTube talks you study with. Each video becomes one **plain markdown file**: slide snapshots at their timestamps, a full timestamped transcript, and editable notes. A small bundled server renders the library as an interactive deep-dive in the browser. No database, no cloud service. Everything is files on disk you fully own.

Architecture (read this first)

The **markdown library is the single source of truth**. The artifact is a thin HTML shell that fetches from the server and writes notes back. Never hardcode video data into the HTML.

• **Library:** a plain folder, set by `VIDEO_LIBRARY_DIR` (default `~/video-deepdives/`).
• One markdown file per video, **filename slug = YouTube id** (e.g. `RtywqDFBYnQ.md`).
• Frontmatter holds video metadata + a `slides` array.
• Body holds the full transcript as `[HH:MM:SS] text` lines.
• `_media/` holds slide images, **namespaced per video** as `<youtube_id>-slide-NN.jpg`

to avoid collisions between videos.

• **Server:** `scripts/serve.py`, a single stdlib + PyYAML file. Start it with:

  python3 scripts/serve.py --dir ~/video-deepdives --port 8000

It serves the artifact at `/` and a small API the artifact talks to:

• `GET /api/video-deepdives` (front page fetches this) lists every video.
• `GET /api/video-deepdives/<id>` returns one video `{meta, body}`.
• `GET /api/video-deepdives/_media/<file>` serves a slide image.
• `PATCH /api/video-deepdives/<id>` with `{fields:{slides:[...]}}` writes notes back.
• **It picks up new videos automatically** the moment a markdown file exists. Adding a video

means writing a markdown file + media; you almost never touch the HTML.

• The `/api/video-deepdives` URL namespace is local to the bundled server.
• **Artifact:** `reference/artifact.html`, served by `serve.py` at `/`. A clean reference copy;

only rewrite it if the user wants a UI change. For new videos, leave it alone.

Requirements

• `yt-dlp` and `ffmpeg` on PATH (download + frame/scene extraction).
• Python 3 with `Pillow` (contact sheet) and `PyYAML` (markdown file + server).

  pip install yt-dlp pillow pyyaml      # ffmpeg via your package manager

Adding a video — the pipeline

All helper scripts are in `scripts/`. Work in a scratch dir (e.g. `/tmp/ytnote-<id>/`), then copy final assets into the library. Set `VIDEO_LIBRARY_DIR` once per shell if you don't want the default. **Do not use em dashes (—) or arrows (→) in notes/titles.**

1. Resolve the id and check embeddability

scripts/setup.sh "<youtube_url_or_id>"

Prints the 11-char `YTID`, the scratch dir, the target library path, and whether YouTube **embedding is allowed** (oembed 200) or **blocked** (oembed 401, e.g. some university talks). If blocked, inline playback won't work but the artifact degrades gracefully to an "open at this moment on YouTube" link, so proceed normally.

2. Download video + subtitles

scripts/download.sh "<YTID>" /tmp/ytnote-<YTID>

Uses `yt-dlp` to grab the video (≤720p is plenty for slide frames) and the best available subtitles (manual if present, else auto-captions) as `.vtt`. Also fetches title/uploader.

3. Detect candidate slide timestamps

scripts/detect_slides.sh /tmp/ytnote-<YTID>/video.mp4 /tmp/ytnote-<YTID>

Runs ffmpeg scene detection (`select='gt(scene,0.3)'`) and writes `scene_times.txt` (seconds). 0.3 is a good default; lower it (0.2) for subtle slide decks, raise it (0.4) for busy video.

4. Build a contact sheet and CURATE

python3 scripts/contact_sheet.py /tmp/ytnote-<YTID>/video.mp4 /tmp/ytnote-<YTID>/scene_times.txt /tmp/ytnote-<YTID>/contact.jpg

Read `contact.jpg` (labeled with index + timestamp). **This is the human-judgment step:** keep frames that are real content slides; **drop talking-head shots, transitions, duplicates, and blurry mid-animation frames.** Save the kept timestamps (seconds) to `/tmp/ytnote-<YTID>/keep.txt`, one per line. Typical talk yields 15-25 slides.

5. Extract the curated slides at full quality and install to _media

python3 scripts/extract_slides.py <YTID> /tmp/ytnote-<YTID>/video.mp4 /tmp/ytnote-<YTID>/keep.txt > /tmp/ytnote-<YTID>/slides.json

Extracts each kept timestamp at 1280px wide, JPEG, and copies them into `$VIDEO_LIBRARY_DIR/_media/` as `<YTID>-slide-01.jpg`, `-02.jpg`, … (numbered in time order). Progress goes to stderr; a clean `slides.json` scaffold prints to **stdout**, so redirect it to a file as shown, then fill in `title` and `note`.

Tip: talks are often a slide + speaker-cam composite, and speakers flip back and forth, so the same slide appears at several timestamps. Keep the cleanest instance of each, and re-anchor each slide's `t` to where it is actually discussed in the transcript (better "play from here" UX).

6. Build the transcript

python3 scripts/vtt_to_transcript.py /tmp/ytnote-<YTID>/*.vtt /tmp/ytnote-<YTID>/transcript.txt

Parses the VTT into clean, de-duplicated `[HH:MM:SS] text` lines (YouT