/measure-before-you-fix

name: measure-before-you-fix
version: 1.0.0
description: |
  Before fixing a slow/stale/timeout alert, measure the step yourself.
  Kill the theory with a stopwatch, not a code change. Measure-first ops
  triage for temporal alerts (stale, timeout, freshness, wedged, N hours
  behind) from gbrain doctor, autopilot, sync, and cron monitors — runs
  BEFORE any timeout raise, threshold change, or pipeline rewrite.
triggers:
  - "keeps timing out"
  - "ETIMEDOUT"
  - "why is this data stale"
  - "freshness alert"
  - "wedged"
  - "job is slow"
  - "sync is stuck"
  - "raise the timeout"
mutating: false
writes_pages: false
writes_to: []
upstream: measure-before-you-fix@fc834ee

Measure Before You Fix

> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) — > before re-deriving a diagnosis, `search` the brain for prior incidents of > the same alert. A recurring alert usually has a recorded verdict already.

Route here on any alert whose claim is **temporal** — "X is stale", "step timed out", "pipeline wedged", "job is slow", "N hours behind". These alerts invite an immediate structural fix (raise the timeout, split the step, reorder the pipeline). Do the measurement first. It is almost always cheaper than the fix, and it frequently invalidates it. (Routing is a harness convention, not a mechanical guarantee — the contract below is the discipline that makes it stick.)

On gbrain surfaces this covers: `gbrain doctor` staleness checks (e.g. sync freshness, cycle freshness), autopilot cycle alerts, the sync stall watchdog (`reason: 'stall_timeout'`), and any cron monitor built on top of them.

The rule

**One stopwatch measurement of the suspect step, before any code change.**

If you cannot state the measured duration of the thing you claim is slow, you do not yet know the root cause, and any fix you write is a guess wearing a diff.

Contract

This skill guarantees:

• No structural fix (timeout raise, step split, pipeline reorder, wrapper

rewrite) is proposed before a measured duration of the blamed step exists.

• The measurement targets the *specific* entity the alert named, not the

aggregate (`--all` hides which member is slow).

• The alert's threshold is compared against the authoritative one

(`gbrain doctor`'s warn/fail lines) before the system is declared unhealthy.

• The verdict explicitly distinguishes "needs more time" from "is wedged" —

they have opposite fixes.

• Read-only: this skill changes no timeouts, thresholds, or code. It produces

a measurement verdict that sizes the fix; the fix itself is a separate, now-informed change.

Procedure

1. **Read the alert's own numbers.** They often contradict the theory already. (`Locks: none` means it isn't lock contention. Note it and drop that branch.)

2. **Time the suspect step directly.** Isolate the smallest unit that the alert blames, and run it with a clock:

   time gbrain sync --source source-a --no-embed

Run it on the *specific* named entity, not the aggregate. A whole-brain run hides which member is slow; `--source source-a` answers the question. Cross-check state with `gbrain sources status` (per-source sync lag).

3. **Compare measured vs. budgeted.** Grep every timeout in the wrapper, not just the default in the helper signature:

   grep -n "timeoutMs\|timeout:" <the wrapper or cron script>

A generous per-call override makes the helper's default irrelevant. Check the call site before blaming the default.

4. **Check the alert threshold against the authoritative one.** Before concluding the system is broken, confirm the alerter and the audit agree on what "bad" means. `gbrain doctor`'s sync-freshness check defaults to 24h warn / 72h fail (env-overridable via `GBRAIN_SYNC_FRESHNESS_WARN_HOURS` / `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`); a cron monitor paging at 12h is speaking below the authoritative warn line. A monitor that *acts* early is correct; a monitor that *speaks* at its act-line is a false-positive generator.

5. **Only now design the fix** — against the number you measured.

The threshold-mismatch failure mode

A cron monitor legitimately acts earlier than the doctor fails, to keep drift out of FAIL territory. That is good design. The bug is reusing the act-threshold as the alert-threshold: everything between "act" and "warn" becomes a recurring page about a healthy system.

**Separate the two constants.** Act at the aggressive line, speak at the authoritative one:

const ACT_HOURS = Number(env.MONITOR_ACT_HOURS || 12);      // act early — fine
const ALERT_HOURS = Math.max(ACT_HOURS, DOCTOR_WARN_HOURS); // speak at the audit's line

Symptom to recognize instantly: **a repeating alert whose numbers sit below the doctor's own warn line**, while the underlying resource looks fine when queried directly.

Red flags that you are theorizing, not diagnosing

• You have a root cause but no measured duration.
• Your fix is a rewrite and you have not run the step once.
• You revise the theory twice without taking a new measurement between revisions.
• The alert says "no recovery in flight" — verify whether recovery is in fact

running before believing it (`ps` for the worker; check the launch flag; `gbrain jobs list` for queued work).

• "Already up to date" in the step output. That step is not your bottleneck.

Anti-Patterns

• **Raising a timeout to fix a stall.** If the step is genuinely hung, a bigger

budget just hangs longer. Measure, then decide between "needs more time" and "is wedged" — they have opposite fixes. (gbrain's sync stall watchdog makes the same distinction natively: it keys on forward progress, not elapsed time.)

• **Rewriting a pipeline on an unmeasured starvation theory.** Splitting steps to

fix starvation that does not exist adds surface area and fixes nothing.

• **Trusting the alert's causal claim.** Alerts report symptoms accurately and

caus