/listening-to-music

name: listening-to-music
description: "Listen to generated music by measuring it and by reading it as sheet music: render Strudel code or record a Web Audio page through the real engine in headless Chromium, then read spectrograms, pYIN pitch lines, chroma, roughness and semitone-rub meters; engrave notes as a score with beat-by-beat harmony; transcribe a melody from audio; read MusicXML/MIDI/ABC or a score image into notes and into Strudel code. Use to check whether generated music sounds right or better (before/after a change), to match a reference track, spectrogram image or score, or to find discordant chords, a buried melody, clipping or muddy mixes. Triggers on 'does it sound better', 'listen to', 'render and check', 'spectrogram', 'sfft/stft', 'discordant', 'clashing chords', 'sounds off', 'compare to the original', 'reproduce this track in Strudel', 'sheet music', 'score', 'notation', 'MusicXML', 'MIDI', 'ABC', 'transcribe', 'OMR'. Pairs with strudeling (writing Strudel); for format conversion use processing-video."
metadata:
  version: 0.2.0

Listening to music

Claude hears nothing, but it can render audio in the real engine and measure it. Set up the loop **render → look → measure → change one thing → re-render**, and keep a table of the numbers per version. A version is better when a number you chose beforehand moves by more than the take-to-take noise, and the spectrogram shows the same thing.

Requirements: Node with `playwright` and Chromium (preinstalled in Claude Code on the web), Python with `librosa soundfile scipy matplotlib pillow`, and for sheet music `music21 verovio mir_eval cairosvg` (`pip install --break-system-packages librosa music21 verovio mir_eval cairosvg`). The first render installs `@strudel/web` into `~/.cache/listening-to-music/`. All scripts live in `scripts/`; run them with `--help` or read the docstring for options.

1. Render

S=/path/to/listening-to-music/scripts
node $S/render_strudel.mjs loop.js --out v1.wav --seconds 40 --warmup 4        # Strudel code
node $S/record_page.mjs page.html --click "#power" --out v1.wav --seconds 40 --warmup 8 \
     --query "station=house" --eval "window.__fm.setSeed(7)"                   # any Web Audio page

Both print peak level and flag clipping. A peak above 1.0 means the listener hears hard clipping; fix levels before judging anything else.

For a before/after comparison, fix the randomness (a seed hook via `--eval`) so both versions play the same material, and record each version twice.

2. Look

python3 $S/spectrogram.py v1.wav --out v1.png --from-onset --pyin-floor G3 --label v1

Then open `v1.png` with the image viewer. The top panel is a note-axis CQT spectrogram with the pYIN lead line, and the bottom panel is chroma. Compare it with the reference picture or the previous version side by side. The `.npz` next to it holds the numbers for step 3.

3. Measure

| Question | Tool | Reads | |---|---|---| | Do simultaneous notes rub? (discordant chords) | `render_strudel.mjs --events` → `clashes.py` | minor 2nd/9th overlap per cycle, by layer pair; worst moments | | Do they rub in the recording? | `rubs.py`, `--pair` for replicated takes | share of tonal peak energy in semitone/minor-9th pairs | | Is it rougher or smoother overall? | `roughness.py`, `--pair` for replicated takes | Sethares roughness median/p90 | | Does it match a reference image? | `reference.py ticks/decode/compare` | per-semitone profile, melody agreement, overtone offsets, chroma agreement | | Is the melody audible over the mix? | `spectrogram.py` pYIN voiced %, `reference.py compare` melody % | |

The clash scan is exact for the notes as written. `rubs.py` confirms them in the audio, where release tails and reverb add overlaps the note data does not show. Record harmony checks with percussion and noise textures muted in both versions: drum partials form their own peaks and hold a full mix at a rub share near 0.28 whatever the chords do. Use `--warmup`, 40 s or more, and two takes per version; `--pair` prints the take-to-take spread and says when a change is inside it.

Summed roughness (`roughness.py`) is a coarse overall measure. On Strudel FM (2026-09-24) it could not tell the harmony fix apart from take-to-take noise in sleep and house. On the same takes, the rub meter measured sleep −51%, ambient −33%, lofi −20% and house −13%, with spreads of 0.006–0.015. A loud bass drone dominates the roughness normalisation and hides a pad's rubs. Don't treat an unmoved roughness figure as proof that nothing changed.

Sheet music: notation out, notation in

Notation is the other way to hear. A score shows voicings, clusters (noteheads pushed sideways are seconds), register and rhythm at a glance, and Claude reads clean engraving accurately. All these scripts share one events JSON (layer, t0, t1, midi; times in bars).

node $S/render_strudel.mjs loop.js --events ev.json --cycles 8          # notes as written
python3 $S/score.py ev.json --out score.png --layers LEAD,KEYS,BASS --harmony   # engrave + analyse
python3 $S/transcribe.py take.wav --out mel.json --bpm 115              # notes as heard (melody)
python3 $S/compare_notes.py ev.json mel.json --ref-layer LEAD --cand-layer melody --align 1
python3 $S/read_score.py tune.musicxml --out ref.json                   # .mxl .mid .abc .krn, corpus:
python3 $S/to_strudel.py ref.json --out tune.js                         # score -> Strudel code

`score.py --harmony` prints each chord change with its pitches, music21's chord name, a Roman numeral in the detected key, and a rub flag. On Strudel FM the same four lofi bars went from 15 of 28 chord changes with a rub to 0 of 28 after the voicing fix, and the engraving showed why: the old Fmaj7 had its E and F side by side.

Measured on 2026-09-25 (compare_notes.py, onset within 0.05 bar and pitch within 50 cents):

| Path | Test | F1 | |---|---|---| | read_score → to_strudel → render events | Bach BWV 66.