/phase-3-analyze

user-invocable: false
install: extras

<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios --> <!-- SPDX-License-Identifier: AGPL-3.0-or-later -->

/pipeline-health — Phase 3: Analyze & Correlate

Phase 3: Analyze & Correlate

3A: Defect category breakdown

**3A-I: Pre-merge defects** — Group review findings (from Phase 2B) by agent prefix.

First, count findings per category from the Phase 2B output:

# Count review findings per agent prefix (SEC, AUTH, BILL, DB, FE, INFRA, API, CONC, SCRP)
# These are the current-period counts to compare against PRIOR_* vars from Phase 1C
COUNT_SEC=$(echo "$FINDINGS_RAW" | grep -c '"SEC' || echo 0)
COUNT_AUTH=$(echo "$FINDINGS_RAW" | grep -c '"AUTH' || echo 0)
COUNT_BILL=$(echo "$FINDINGS_RAW" | grep -c '"BILL' || echo 0)
COUNT_DB=$(echo "$FINDINGS_RAW" | grep -c '"DB' || echo 0)
COUNT_FE=$(echo "$FINDINGS_RAW" | grep -c '"FE' || echo 0)
COUNT_INFRA=$(echo "$FINDINGS_RAW" | grep -c '"INFRA' || echo 0)
COUNT_API=$(echo "$FINDINGS_RAW" | grep -c '"API' || echo 0)
COUNT_CONC=$(echo "$FINDINGS_RAW" | grep -c '"CONC' || echo 0)
COUNT_SCRP=$(echo "$FINDINGS_RAW" | grep -c '"SCRP' || echo 0)
# ISSUE_SPEC from Phase 2L audit findings
COUNT_ISSUE_SPEC=$ISSUE_SPEC_COUNT

TOTAL_FINDINGS=$((COUNT_SEC + COUNT_AUTH + COUNT_BILL + COUNT_DB + COUNT_FE + COUNT_INFRA + COUNT_API + COUNT_CONC + COUNT_SCRP))

# Compute trend for each category using _trend helper from Phase 1C
TREND_SEC=$(       _trend "$COUNT_SEC"        "$PRIOR_SEC")
TREND_AUTH=$(      _trend "$COUNT_AUTH"       "$PRIOR_AUTH")
TREND_BILL=$(      _trend "$COUNT_BILL"       "$PRIOR_BILL")
TREND_DB=$(        _trend "$COUNT_DB"         "$PRIOR_DB")
TREND_FE=$(        _trend "$COUNT_FE"         "$PRIOR_FE")
TREND_INFRA=$(     _trend "$COUNT_INFRA"      "$PRIOR_INFRA")
TREND_API=$(       _trend "$COUNT_API"        "$PRIOR_API")
TREND_CONC=$(      _trend "$COUNT_CONC"       "$PRIOR_CONC")
TREND_SCRP=$(      _trend "$COUNT_SCRP"       "$PRIOR_SCRP")
TREND_ISSUE_SPEC=$(_trend "$COUNT_ISSUE_SPEC" "$PRIOR_ISSUE_SPEC")

_pct() { [ "$TOTAL_FINDINGS" -gt 0 ] && echo "scale=0; $1 * 100 / $TOTAL_FINDINGS" | bc || echo "0"; }

Then render the table using the computed values:

| Category | Count | % of Total | Trend vs Prior Period | |----------|-------|------------|---------------------| | SEC | $COUNT_SEC | $(_pct $COUNT_SEC)% | $TREND_SEC | | AUTH | $COUNT_AUTH | $(_pct $COUNT_AUTH)% | $TREND_AUTH | | BILL | $COUNT_BILL | $(_pct $COUNT_BILL)% | $TREND_BILL | | DB | $COUNT_DB | $(_pct $COUNT_DB)% | $TREND_DB | | FE | $COUNT_FE | $(_pct $COUNT_FE)% | $TREND_FE | | INFRA | $COUNT_INFRA | $(_pct $COUNT_INFRA)% | $TREND_INFRA | | API | $COUNT_API | $(_pct $COUNT_API)% | $TREND_API | | CONC | $COUNT_CONC | $(_pct $COUNT_CONC)% | $TREND_CONC | | SCRP | $COUNT_SCRP | $(_pct $COUNT_SCRP)% | $TREND_SCRP | | ISSUE_SPEC | $COUNT_ISSUE_SPEC | $(_pct $COUNT_ISSUE_SPEC)% | $TREND_ISSUE_SPEC |

**If PRIOR_* vars are all empty (baseline run)**: all trend cells show "—" (no prior period to compare against).

**Note**: ISSUE_SPEC findings come from `/audit` (`audit-finding` label + body contains "ISSUE_SPEC"), not from review agents. They represent upstream failures — bad issue specs that the pipeline followed correctly but that produced wrong or wasted work.

**3A-II: Post-deploy defects** — Group audit findings (from Phase 2L) by failure point:

| Failure Point | Count | % of Total | Meaning | |---------------|-------|------------|---------| | INVESTIGATION | ? | ?% | Wrong verdict or missed scope — root cause misdiagnosed | | IMPLEMENTATION | ? | ?% | Code written incorrectly or incompletely | | REVIEW | ? | ?% | Review agents missed the issue or wrong agents triggered | | REVIEW_FALSE_NEG | ? | ?% | Review agent saw area but approved a defect (escape signal) | | QUALITY_GATE | ? | ?% | Static check should have caught this pattern but didn't | | DEPLOY_GATE | ? | ?% | Deploy-info should have flagged risk but didn't | | CONTEXT | ? | ?% | Missed relevant prior work or related code paths | | ARCHITECT | ? | ?% | Missed code paths or wrong implementation plan | | ORCHESTRATION | ? | ?% | Wave ordering wrong or dependency missed | | ISSUE_SPEC | ? | ?% | Bad issue spec — pipeline followed it correctly, spec was wrong | | OTHER | ? | ?% | Uncategorized or compound failure |

**Cross-signal analysis**: Compare 3A-I (pre-merge review findings) with 3A-II (post-deploy audit findings) to identify where the pipeline gates are misaligned. High REVIEW_FALSE_NEG in 3A-II with low REVIEW findings per PR in 3A-I suggests review agents are approving too liberally rather than finding real defects.

**Identify the top 3 defect categories across both tables.** These are where the builder/review is weakest and where pipeline improvements would have the highest ROI.

3B: Prompt change impact

For each Forge commit in the window: 1. What command was changed? 2. What was the intent? (read the commit message) 3. Did the target metric improve after this change?

• If the change was to `review-pr-agents.md` → did false positive rate change?
• If the change was to `work-on.md` Step 3G → did build failure rate change?
• If the change was to `orchestrate.md` → did milestone merge success rate change?

**Caution**: Correlation ≠ causation. Note confounding factors (e.g., milestone size, new developers).

3C: Compute health score

Health Score = weighted average of:
  Pre-merge quality:
  - Build pass rate (staging→main):         weight 12%    target 95%+
  - Review findings per PR:                 weight 9%     target < 0.5
  - Manual fix-up rate:                     weight 8%     target < 15%
  - False positive rate:                    weight 6%     target < 10%
  - Pipeline self-correction rate:          weight 7%     target < 15%

  Post-deploy quality:
  - Post-deploy failure rate (Phase 2L):    weight 9%     target < 5%
  - Review escape rate (Phase 2L):          weight 9%     target < 2%

  Pipeline