/mulmoterminal-video-edit

name: mulmoterminal-video-edit
description: Edit a video by talking about it — cut out a part, join clips, change the speed, even out or replace the sound, make it vertical — and add subtitles made from what is said in it, or captions and titles placed on screen (telops). Uses ffmpeg in the terminal and whisper.cpp for the speech, writes a new file beside the original, and opens the result in the pane on the right of the terminal (showInPane). Use when the user asks to trim, cut, join, speed up, resize, subtitle, caption, translate the subtitles of, or put text on a video. The user writes no commands. Works in whatever language the user writes in.

Edit a video

The user points at a video and says what they want in plain words. They do not write ffmpeg commands; you do, in the terminal, and show them the result. Talk to them about their video — the part from 0:12 to 0:40, the subtitles, the title at the start — not about codecs and filters, unless they ask.

What you need

• `ffmpeg` and `ffprobe`. Check with `ffmpeg -version`. If it is missing, say so and give the

install command (`brew install ffmpeg` on a Mac); do not install it yourself.

• For subtitles or captions drawn onto the picture, ffmpeg built with libass. Check:

`ffmpeg -hide_banner -filters | grep -E ' (subtitles|ass) '` prints two lines when it is.

• For subtitles made from speech, `whisper-cli` from whisper.cpp (`brew install whisper-cpp` on a

Mac; the same package MulmoTerminal's voice input uses) and a model. Look for one MulmoTerminal has already downloaded, in the workspace's `models` folder:

  ls "${CLAUDE_CWD:-$HOME/mulmoclaude}/models"/ggml-*.bin

Use the file you find (prefer `ggml-large-v3-turbo.bin` when there are several) as `$model` below. If there is none, ask before downloading one, and say how big it is: `ggml-large-v3-turbo.bin` (about 1.6 GB, the most accurate) or `ggml-small.bin` (about 490 MB, faster, less accurate). Then put it in that folder, so voice input finds it too:

  models="${CLAUDE_CWD:-$HOME/mulmoclaude}/models" && mkdir -p "$models" &&
  curl -L -o "$models/ggml-large-v3-turbo.bin" https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin

(`ggml-small.bin` the same way.) Then `$model` is the file you downloaded.

• `showInPane`, to open the result beside the terminal. In a project directory it exists only when

**Canvas** was switched on before the session started. Without it, give the user the file's path.

The speech is transcribed on this machine. Never send the video or its sound anywhere.

Rules for every edit

• **Never change the original.** Write a new file in the same folder, named after the original and

the edit: `talk.mp4` → `talk-cut.mp4`, `talk-subtitled.mp4`. If that name is taken, add a number. When the original is outside your working directory, write the new file in your working directory instead, so it can be shown.

• Keep intermediate files in a working directory of their own (`work=$(mktemp -d)`), and give them

plain names (`audio.wav`, `subs.srt`), which keeps the filter arguments below free of quoting.

• Encode the result so it plays everywhere, the pane included: H.264 video, AAC sound,

`yuv420p`, and the index at the front of the file.

  ENC=(-c:v libx264 -crf 20 -preset medium -pix_fmt yuv420p -c:a aac -b:a 192k -movflags +faststart)

This is `"${ENC[@]}"` below. Keep it an array and quoted that way: a plain string (`ENC="…"`) reaches ffmpeg as one argument in zsh, and the command fails. Re-encoding takes time — roughly the length of the video or longer on a laptop — so tell the user before a long one, and for a style they have not seen yet, make a short sample first (add `-t 20` for the first twenty seconds) and show that.

1. Look at the video first

ffprobe -v error -show_entries format=duration,size:stream=index,codec_type,codec_name,width,height,r_frame_rate,channels -of json "talk.mp4"

Tell the user its length, picture size, frame rate, and whether it has sound. Times in what they ask for ("from the 12th second", "the last minute") are measured against this.

2. Cutting, joining and the rest

**A part** (accurate to the frame, by re-encoding):

ffmpeg -ss 00:00:12 -to 00:00:40 -i talk.mp4 "${ENC[@]}" talk-cut.mp4

`-c copy` instead of `"${ENC[@]}"` is instant but can only cut at a keyframe, so the start may be off by a few seconds. Offer it only for a long video where that does not matter, and say so.

**Several parts kept, the rest dropped**: cut each part as above into the working directory, then join them.

**Joining** clips made the same way (same size, frame rate and codecs, such as parts of one video):

printf "file '%s'\n" part1.mp4 part2.mp4 > list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy joined.mp4

Clips from different sources first need the same size and frame rate; join them with the `concat` filter instead, scaling each to the first one's size:

ffmpeg -i a.mp4 -i b.mp4 -filter_complex "[0:v]scale=1280:720,setsar=1,fps=30[v0];[1:v]scale=1280:720,setsar=1,fps=30[v1];[v0][0:a][v1][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" "${ENC[@]}" joined.mp4

A clip with no sound has no `[n:a]`, and the join fails. Add silence as one more input, as long as that clip, and use it in the clip's place: with `b.mp4` (4 s, silent) as input 1, add `-f lavfi -t 4 -i anullsrc=r=48000:cl=stereo` as input 2 and write `[v1][2:a]` for it.

**Speed**: `-filter:v "setpts=PTS/1.5" -filter:a "atempo=1.5"` plays at 1.5×. `atempo` takes 0.5 to 2; chain it for more (`atempo=2,atempo=1.5` is 3×).

**Sound**: even out the loudness with `-af loudnorm=I=-16:TP=-1.5:LRA=11`; remove it with `-an`; replace it with music, cut to the video's length:

ffmpeg -i talk.mp4 -i music.mp3 -map 0:v -map 1:a -shortest "${ENC[@]}" talk-music.mp4

Music under the speech instead: `-filter_complex "[1:a]volume=