/radar-self-eval

name: radar-self-eval
description: |
  Compute the radar's self-calibration: weekly funnel metrics (queue dynamics, exploration compliance, off-axis rate), a monthly hit/miss retrospective against what actually became big, and up to 3 curator proposals. Use during every weekly recalibration, after the source-strategy review; results go into the append-only `logs/calibration.md` and into the weekly report.

Self-evaluation

The radar must measure whether it is winning, not assume it. Three parts: metrics (every week), retrospective (monthly), curator proposals (every week).

1. Weekly metrics

Compute from the week's daily reports (primary source — they list ledger changes) and, where needed, `git log -p -- TRENDS.md` since the previous weekly commit:

• **queue funnel**: items added / promoted to trend / dropped / older than 14

days and still unverified (stale)

• **ledger**: evidence items added, stage moves (up and down)
• **exploration compliance**: daily runs whose `logs/source_rotation.md` line

contains a venue-exploration entry ÷ daily runs executed

• **off-axis rate**: share of new queue items that do NOT match any axis in

`logs/strategy_notes.md` (judgment call — name them)

• **discovery lag**: for each evidence item added this week, the days between

its evidence-line date and the date it entered the ledger (commit date via `git log -p -- TRENDS.md`, or the daily reports). Report the median, split by channel — exploration finds vs queue promotions (backfill) — plus the backfill share of all new evidence. This is the daily-ness KPI.

• **coverage** (the self-check that auto-detects the listed-but-never-swept failure class):

for each "swept every run" heading in SOURCES.md (lab blogs, YouTube curators, pointer/digest blogs, discovery venues, tool-discovery channels), enumerate EVERY entry under it — INCLUDING bullets nested under a sub-label (e.g. a "non-GitHub channels:" sub-block), not just top-level list entries — and diff that full list against the week's `logs/source_rotation.md` lines, classifying every listed source as `opened`, `degraded`, or **MISSING** (in SOURCES.md but never in any log line this week). MISSING = a coverage lie — NAME them; never report "0 missing" without having run the full nested list-vs-log diff. A source MISSING or `degraded` for the whole week is a heal-or-REMOVE candidate (see Amendments): the registry must be honest about what it actually sweeps. AMENDMENT (2026-07-11, W28, applied — proposed W27 as Proposal D, signal persisted): "opened" is not enough on its own — when SOURCES.md documents a CONFIRMED heal for a source (a specific working method, e.g. a search+extract pipeline replacing a broken feed), check whether the daily logs actually RAN that method this week, not just re-logged the source's OLD pre-heal `degraded:` label. A source can pass "opened" for weeks while quietly reading only a stale/broken slice of its own surface (multi-section sources like Anthropic /research+/news are the same failure class as an unused heal) — name any source where this happened as its own finding, distinct from a plain MISSING.

• **routing-leak**: queue items tagged with (or clearly on) an EXISTING trend's

axis yet absent from that trend's evidence — on-axis primaries hoarded in the queue instead of routed to evidence. Count and name them; each is a routing miss to correct this week.

• **source-discovery**: from SOURCES.md → "Discovered-source candidates" and the

week's reports — candidates STAGED this week / PROMOTED to a swept source / PRUNED, plus discovery-latency = runs (or days) between an org's FIRST appearance in the staging tally and its promotion. The tool-author/vendor analog of routing-leak: a recurring on-axis org that stays un-promoted (≥2 weeks in the tally) is a coverage leak — NAME it and promote-or-drop it now. Zero candidates staged for two weeks while new evidence keeps arriving from the SAME few orgs is itself a signal the capture-the-source step isn't firing.

• **capture-leak** (the notes-prose blind spot, complementary to routing-leak):

scan every trend's `notes` field AND the week's reports for named primaries (arXiv ids, repo / release URLs) that appear NOWHERE as a discrete `observation_queue` item or an evidence line on any trend. Each is a capture leak — a primary the radar named but never captured as a first-class item, so it never reached study-pick selection or the convergence check. NAME them and queue them now. routing-leak only scans the queue, so an item that never reached the queue is invisible to it; this metric is what catches it.

Append ONE dated line to `logs/calibration.md` (the externalized self-eval log; the `## calibration` section of TRENDS.md is now only a pointer): `- YYYY-MM-DD — W<nn>: queue +a/→p/−d/stale s · evidence +e · moves m · exploration c/r · off-axis o/a · lag expl Xd / backfill Yd (Z%) · coverage <opened>/<listed> (miss <n>, degr <d>) · routing-leak <n> · capture-leak <n> · src-disc <staged>/<promoted> (lat L)` and include the same numbers, readable, in the weekly report.

Interpretation thresholds (act, don't just log):

• exploration compliance < 100% → call it out; two weeks running → propose a

daily prompt/skill refinement.

• stale > half the queue → scanning breadth exceeds verification capacity:

propose narrowing or a verification-only day.

• off-axis = 0 for two consecutive weeks → anchoring warning, even if the

tunnel-vision check passed.

• exploration-channel median lag > 7 days → the venues are stale: rotate them.
• backfill share not shrinking for two consecutive weeks (once the bootstrap

queue is cleared) → the radar is doing literature review, not daily detection: propose rebalancing scan time toward exploration.

2. Monthly retrospective (first weekly run of the month: `date +%d` ≤ 7)

Goal: ground truth — did the radar see early what later became big?

1. Pick 2–3 items that