/radar-source-heal

name: radar-source-heal
description: |
  Repair a source or tool that keeps failing — a broken/moved/stale feed, a 404, an empty result, a parse/JSON error — instead of logging "degraded" forever. Diagnose the failure class, find a working access path, and record the repaired method in SOURCES.md so future runs don't repeat it. Generalizes to any future source/tool breakage. Use whenever the lab sweep, pulse, or repo-watch hits a source that failed the same way before.

Source self-healing

Logging "degraded" run after run is a failure of the radar, not of the source. If you logged the same breakage last run (check the recent tail of `logs/source_rotation.md` and SOURCES.md notes), **fix it this run** — don't just re-log it.

The heal worklist comes from the log (don't guess)

Coverage-honesty means every sweep logs each source as `opened` or `degraded: <reason>` in `logs/source_rotation.md`. So the heal worklist is a precise, queryable set — you do not guess what to fix:

• `grep 'degraded:' logs/source_rotation.md | tail` → the candidates. A source

marked `degraded` in 2+ recent runs (or obviously degraded NOW — empty feed, 404, JSON/parse error, a feed whose newest item is months old) is OWED a heal this run. One transient blip → log it, retry next run; don't heal on a single failure.

• A source that is in a SOURCES.md "swept every run" list but is MISSING from the

log entirely (neither opened nor degraded — a coverage lie) is also a heal target: it is usually being skipped because it has no recorded access method and falls out of the list-iteration (this is the alphamatch failure class). Find a working method and record it.

The fix is written back into SOURCES.md (below), so the NEXT list-iteration consumes it automatically — that is what closes the loop: iterate → degraded / missing → heal → SOURCES.md method → next iteration succeeds.

Diagnose the failure class, then repair

**Class A — broken / moved / stale feed or page**

• Open the source's HTML index. Find the real feed from the page's

`<link rel="alternate" type="application/rss+xml|atom+xml">`, or try common paths: `/rss`, `/feed`, `/rss.xml`, `/atom.xml`, `/index.xml`, `/blog/feed.xml`.

• Moved blog (org rebranded/changed domain) → search for the current URL and

confirm it resolves with recent posts.

• Stale feed (newest post months old while the org is clearly still active) →

the org now posts elsewhere; find the live channel (a different domain, their HF org activity, a docs/changelog page) and switch to it.

• No feed exists at all → set the method to "HTML index via Tavily extract" and

note it, so the sweep stops hunting for a feed.

• **Same URL failing 2+ sessions running → try an alternate URL PATH on the same

domain before concluding the source is unreachable** (AMENDED 2026-07-25, W30 Proposal J — motivated by Bishop Fox: W27/W28 marked it unresolved after repeatedly retrying one individual post URL, when the plain `/blog` index path worked the whole time). A rendering/JS failure on one path does not mean the whole domain fails — try the bare blog/research index, `/blog`, `/research`, `/news`, or a category root before the domain is written off as JS-blocked or dead.

**Class B — failing tool call on an otherwise-live source**

• Vary the call shape: different `tvly` flags, `extract` vs `search`,

`--extract-depth advanced`, smaller/larger scope.

• Prefer the source's own structured endpoint when it has one (many sites expose

a JSON/search API — e.g. a public search endpoint — that returns clean data without scraping).

• Fall back to the built-in web tools if Tavily keeps failing on that source.
• Find the shape that returns clean, parseable data.

A search is not a feed (recency is not guaranteed)

A topic SEARCH (`tvly search "<terms>" --include-domains <site>`) returns what RANKS, not what is NEWEST. Used as the access method for a feed-less or JS-rendered site it produces confident false negatives — "checked, nothing fresh" while the site published on-scope posts the query did not rank. A search is fine as an EXTRACT aid; it can never be the CHECK that discharges a coverage promise. Heal such a source with a method that ENUMERATES what the site published, in this order: 1. its feed (find it as in Class A), else 2. the sitemap diff (`sitemap_diff.py <host>`, bundled with the sweep skill: URLs new since the last run), else 3. the index diff (`sitemap_diff.py index:<section index url>`: the article links in the index page's raw HTML), else 4. a rendered copy of the index (`tvly extract`, or a reader proxy) diffed against the URLs already in the repo. Only if all four fail is the source search-only: log it `search-only (recency not guaranteed)` — never `opened`/`checked` — and keep it on the heal worklist. Re-check every "healed" source whose recorded method is a search: it was never actually healed.

Record the fix (this is the durable part)

Write the WORKING access method back into `SOURCES.md` next to that source, dated, REPLACING the "degraded/404/stale" note — e.g. "feed at <url> (confirmed working YYYY-MM-DD)" or "use <endpoint>, not the feed (YYYY-MM-DD)". The registry is agent-owned; the repaired method is what makes the fix permanent. If a source is genuinely dead with no replacement, mark it `RETIRED <date> — <reason>` and drop it from the active list.

Persistent environmental / IP blocks (don't re-log forever)

Some sources are blocked not by a broken path but by the environment — a datacenter-IP block by the site, or egress restricted to a few hosts. You cannot "repair" these, but you must NOT re-log "blocked from this IP" every run either.

• **Diagnose** which it is: try a neutral *programmatic* endpoint (an API host

like `hn.algolia.com`). If it works, the block is site-specific → switch that source to its API host (e.g. HN → `hn.algolia.com/api/v1/...`; HF → the `huggingface.co/api/...` endpoints; Reddit → the `.js