upgrade-failure-modes

Upgrade Orchestration Failure Modes Reference

> **Scope**: Common orchestration failures in the 6-phase system-upgrade workflow: premature implementation, approval gate bypass, inline edits, and scope creep. Covers detection and remediation. > **Version range**: system-upgrade-engineer, all versions > **Generated**: 2026-04-15

---

Overview

The system-upgrade workflow has mandatory gates and specialist dispatch rules. The most damaging failures are silent: implementing without user approval at Phase 3, making domain changes inline instead of delegating to specialists, and running full-repo audits for scoped changes. These produce either unauthorized bulk edits or subtly incorrect results that bypass domain validation.

---

<!-- no-pair-required: section header, not a standalone failure mode block -->

Failure Mode Catalog

Implementing Without Phase 3 Approval

**Detection**:

# Check if task_plan.md has a PLAN section followed immediately by IMPLEMENT
grep -n "^## Phase 3\|^## Phase 4\|PLAN\|IMPLEMENT" task_plan.md
# If Phase 4 timestamp precedes user approval acknowledgment, gate was skipped

# Check git log for commits that skip the plan-present step
git log --oneline --since="1 hour ago" | grep "chore/system-upgrade"

**What it looks like**: Moving from Phase 2 AUDIT directly to Phase 4 IMPLEMENT without presenting the ranked table and waiting for a response.

**Why wrong**: Users lose control of what changes in their system. Bulk edits to governance infrastructure (hooks, routing tables, agent frontmatter) are hard to reverse and affect every subsequent session. The approval gate exists specifically because the agent cannot know which changes the user wants to prioritize or defer.

Do instead: Present the Phase 3 table (Tier | Component | Change Type | Effort | Group) and wait for explicit acknowledgment before any writes.

---

Implementing Domain Changes Inline

**What it looks like**: Directly editing `hooks/posttool-rename-sweep.py` instead of dispatching `hook-development-engineer`. Writing new agent frontmatter inline instead of dispatching `skill-creator`.

Do instead: dispatch `hook-development-engineer` for hook changes, `skill-creator` for agent and skill changes, and `routing-table-updater` for routing table changes. Details follow.

**Detection**:

# Look for direct file edits to domain files in agent output logs
grep -rn "Edit\|Write" hooks/*.py agents/*.md --include="*.py" --include="*.md" 2>/dev/null
# system-upgrade-engineer should only create task_plan.md and branch setup files

**Why wrong**: Domain specialists (hook-development-engineer, skill-creator) carry template conventions, event schema knowledge, and frontmatter validation that inline edits bypass. A hook written inline without hook-development-engineer's exit code contract knowledge will likely use wrong exit codes. An agent written inline will miss required frontmatter fields.

Do instead:

• Hook changes → dispatch `hook-development-engineer`
• Agent/skill changes → dispatch `skill-creator`
• Routing table changes → dispatch `routing-table-updater` skill
• Only create `task_plan.md` and branch setup files directly

---

Not Scoping the Audit to Signal-Identified Components

**Detection**:

# Check if audit scanned all components for a targeted change
grep -c "Scanned\|checked\|audited" task_plan.md
# An audit for 2-hook changes should reference < 20 components, not 120+

# Check signal column presence in Change Manifest
grep "Component Types\|component type" task_plan.md
# Should be present — if missing, audit had no scope

**Why wrong**: Auditing all 120+ skills for a 2-hook change produces noise proportional to scope. When every component appears in the audit, the PLAN phase cannot distinguish affected from unaffected. Tier assignment degrades to noise.

Do instead: Build the Change Manifest with a "Component Types" column first. Default scope is 10 most-recently-modified agents + all hooks + affected routing tables. Comprehensive audit only with the explicit "comprehensive" keyword from the user.

---

Skipping Validation Scoring

**Detection**:

# Check if VALIDATE phase ran agent-evaluation
grep -n "agent-evaluation\|before.*score\|after.*score" task_plan.md
# Should appear in Phase 5 section; if absent, validation was skipped

**What it looks like**: Creating PR directly after IMPLEMENT without running `agent-evaluation` on modified components.

**Why wrong**: An agent that scores lower after modification has regressed. Without the before/after delta, regressions are invisible until users report breakage. The upgrade pipeline exists to *improve* quality, not maintain it.

Do instead: Run `agent-evaluation` on each modified component. Report the numeric delta. If any component scores lower, surface it to the user and keep the modified component in place until they choose a rollback. Do not downgrade the regression as "necessary."

---

Force-Pushing or Committing to Main

**Detection**:

# Confirm current branch is not main before any writes
git branch --show-current
# Should NEVER be main during an upgrade run

# Check no force-push flags in recent git commands
history | grep "push --force\|push -f"

**Why wrong**: Commits to main bypass branch protection and review. Force-push to main overwrites upstream state and is unrecoverable without a backup. The branch naming convention (`chore/system-upgrade-YYYY-MM-DD`) exists to ensure all changes go through PR.

Do instead: Run `git checkout -b chore/system-upgrade-YYYY-MM-DD` before Phase 4. Never use `--force` or `-f` on push. If already on main, stash and create branch before proceeding.

---

Reporting Regression as "Necessary"

**Detection**:

# Check VALIDATE section for score drops paired with justification phrases
grep -A 3 "score.*lower\|regressed\|dropped" task_plan.md | grep -i "necessary\|intentional\|expected\|trade-off"
# These phrases in the sa