/qa

description: Run quality assessment on a SpecWeave increment with risk scoring and quality gate decisions

sw:qa - Quality Assessment Command

**IMPORTANT**: You MUST invoke the CLI `specweave qa` command using the Bash tool. The slash command provides guidance and orchestration only.

Purpose

Run comprehensive quality assessment on an increment using:

• ✅ **Gate 1: Rule-based validation** (130+ automated checks)
• ✅ **Gate 2: LLM-as-Judge** (AI quality assessment with chain-of-thought reasoning)
• ✅ **Gate 3: Risk scoring** (BMAD Probability × Impact quantitative assessment)
• ✅ **Quality gate decisions** (PASS/CONCERNS/FAIL)

LLM-as-Judge Pattern

This command implements the **LLM-as-Judge** pattern - an established AI/ML evaluation technique where an LLM evaluates outputs using structured reasoning.

**How it works:**

┌─────────────────────────────────────────────────────────────┐
│                    LLM-as-Judge Gate                        │
├─────────────────────────────────────────────────────────────┤
│  Input: spec.md, plan.md, tasks.md                         │
│                                                             │
│  Process:                                                   │
│  1. Chain-of-thought analysis (7 dimensions)               │
│  2. Evidence-based scoring (0-100 per dimension)           │
│  3. Risk identification (BMAD P×I formula)                 │
│  4. Formal verdict (PASS/CONCERNS/FAIL)                    │
│                                                             │
│  Output: Structured quality report with:                   │
│  - Blockers (MUST fix)                                     │
│  - Concerns (SHOULD fix)                                   │
│  - Recommendations (NICE to fix)                           │
└─────────────────────────────────────────────────────────────┘

**Why LLM-as-Judge?**

• **Consistency**: Applies uniform evaluation criteria
• **Depth**: Catches nuanced issues humans might miss
• **Speed**: ~30 seconds vs hours of manual review
• **Documented reasoning**: Explains WHY something is an issue

Usage

sw:qa <increment-id> [options]

Examples

# Quick mode (default)
sw:qa 0008

# Pre-implementation check
sw:qa 0008 --pre

# Quality gate check (comprehensive)
sw:qa 0008 --gate

# Export blockers to tasks.md
sw:qa 0008 --export

# CI mode (exit 1 on FAIL)
sw:qa 0008 --ci

# Skip AI assessment (rule-based only)
sw:qa 0008 --no-ai

# Force run even if rule-based fails
sw:qa 0008 --force

Options

• `--quick` - Quick mode (default) - Fast assessment with core checks
• `--pre` - Pre-implementation mode - Check before starting work
• `--gate` - Quality gate mode - Comprehensive check before closing
• `--full` - Full multi-agent mode (Phase 3)
• `--ci` - CI mode - Exit 1 on FAIL (for automation)
• `--no-ai` - Skip AI assessment - Rule-based validation only (free, fast)
• `--export` - Export blockers/concerns to tasks.md
• `--force` - Force run even if rule-based validation fails
• `-v, --verbose` - Show recommendations in addition to blockers/concerns

What It Does

Step 1: Rule-Based Validation (Always First, Always Free)

The command runs 120+ validation checks on increment files:

• ✅ File existence (spec.md, plan.md, tasks.md)
• ✅ YAML frontmatter structure
• ✅ AC-ID traceability (spec.md → tasks.md)
• ✅ Link integrity
• ✅ Format consistency

**If rule-based fails** → Stop (don't waste AI tokens) unless `--force` flag used

Step 2: AI Quality Assessment (Optional, skip with `--no-ai`)

**IMPORTANT**: This step uses the `increment-quality-judge-v2` **skill** (auto-activated).

The skill provides guidance and the CLI handles execution:

# CLI invokes quality assessment directly
specweave qa 0008 --pre

**DO NOT spawn agents for quality assessment** - use the CLI command which handles everything internally.

The assessment evaluates:

• **7 Dimensions**:

1. Clarity (18% weight) 2. Testability (22% weight) 3. Completeness (18% weight) 4. Feasibility (13% weight) 5. Maintainability (9% weight) 6. Edge Cases (9% weight) 7. **Risk Assessment (11% weight)**

**Risk Assessment** uses quantitative method:

• Probability (0.0-1.0) × Impact (1-10) = Risk Score (0.0-10.0)
• 4 categories: Security, Technical, Implementation, Operational
• Severity: CRITICAL (≥9.0), HIGH (6.0-8.9), MEDIUM (3.0-5.9), LOW (<3.0)

Step 3: Quality Gate Decision

Based on thresholds:

**FAIL** if any:

• Risk score ≥ 9.0 (CRITICAL)
• Test coverage < 60%
• Spec quality < 50
• Critical security vulnerabilities ≥ 1

**CONCERNS** if any:

• Risk score 6.0-8.9 (HIGH)
• Test coverage < 80%
• Spec quality < 70
• High security vulnerabilities ≥ 1

**PASS** otherwise

Step 4: Display Report

Show results with:

• 🟢 PASS / 🟡 CONCERNS / 🔴 FAIL decision
• Blockers (MUST fix)
• Concerns (SHOULD fix)
• Recommendations (NICE to fix, with `--verbose`)
• Spec quality scores (7 dimensions)
• Summary (duration, tokens, cost)

Step 5: Export (Optional)

If `--export` flag provided:

• Append blockers/concerns to tasks.md
• Add priority (P0 for blockers, P1 for concerns)
• Include mitigation strategies

Implementation

**When user runs `/qa <increment-id>`**:

1. **Parse and normalize arguments**

   let incrementId = args[0]; // e.g., "0008" or "0008-feature-name"

   // Normalize increment ID
   if (incrementId.includes('-')) {
     // Extract numeric portion: "0008-feature-name" → "0008"
     incrementId = incrementId.split('-')[0];
   }
   // Convert to 4-digit format: "8" → "0008"
   incrementId = incrementId.padStart(4, '0');

   const options = parseOptions(args.slice(1));

Both formats work: `sw:qa 0153` or `sw:qa 0153-feature-name`

2. **Invoke CLI command via Bash tool**

   specweave qa 0008 --pre --export

3. **CLI handles everything**:

• Rule-based validation
• AI assessment invocation
• Quality gate decision
• Rep