/verification-quality

name: verification-quality
description: |
  Check code quality and change risk with ruflo's shipped analysis commands (analyze diff, analyze complexity, analyze code), and run or extend ruflo's own CI regression guards. Use when assessing a change's risk, finding complexity hot spots, or adding a regression guard to ruflo. The truth-scoring and auto-rollback commands described later in this file are design notes, not shipped CLI commands.

Verification & Quality Assurance Skill

What This Skill Does

This skill provides a comprehensive verification and quality assurance system that ensures code quality and correctness through:

• **Truth Scoring**: Real-time reliability metrics (0.0-1.0 scale) for code, agents, and tasks
• **Verification Checks**: Automated code correctness, security, and best practices validation
• **Automatic Rollback**: Instant reversion of changes that fail verification (default threshold: 0.95)
• **Quality Metrics**: Statistical analysis with trends, confidence intervals, and improvement tracking
• **CI/CD Integration**: Export capabilities for continuous integration pipelines
• **Real-time Monitoring**: Live dashboards and watch modes for ongoing verification

> **Shipped vs. aspirational.** The *concrete, in-CI* verification stack — the 6 regression-guard jobs + the witness manifest + the tool-discoverability audit — is real and runs on every push. The truth-scoring / auto-rollback / WebSocket-dashboard surface described later in this doc is partly shipped (`ruflo verify` runs the witness checks) and partly design — treat the "CI Guards" section below as the authoritative current state.

CI Guards — what's actually shipped (current state)

Ruflo's regression protection is three layers, all gated before publish. Authoritative reference: [`verification/README.md`](../../../verification/README.md).

| Layer | What | CI job(s) in `.github/workflows/v3-ci.yml` | ADR | |---|---|---|---| | **1 — install/behavioral smoke** | Exercise user-visible failure modes against a real build | `smoke-install-no-bsqlite` (npm install on platforms w/o prebuilds), `plugin-hooks-smoke` (#1859/#1862 — hook flag parsing), `mcp-protocol-smoke` (#1874 — HTTP MCP wire format), `memory-import-smoke` (#1883/#1884 — WSL path + key sanitization), `mcp-roundtrip-smoke` (#1889 paired-tool round-trip + #1863 cli-no-crash + ADR-095 G2 consensus-transport) | ADR-102 | | **1 — discoverability gate** | Every MCP tool description must answer "use this over native when?" | `tool-descriptions-audit` — `scripts/audit-tool-descriptions.mjs`, baseline at `verification/mcp-tool-baseline.json` (monotone-decreasing: noGuidance / tooShort / duplicates) | ADR-112 | | **2 — cryptographic witness** | Every documented fix's load-bearing marker must still be present in dist; Ed25519-signed, per-OS bundles | `witness-verify` (ubuntu/macos/windows) — `plugins/ruflo-core/scripts/witness/verify.mjs` against `verification/<os>/manifest.md.json` | ADR-103 | | **3 — temporal history** | When was a regression introduced | `verification/<os>/history.jsonl` + `history.mjs` (`summary` / `regressions` / `timeline`) | ADR-103 |

Run the guards locally

# Tool-description discoverability audit (ADR-112)
node scripts/audit-tool-descriptions.mjs                       # fails if any baseline count rises
node scripts/audit-tool-descriptions.mjs --update-baseline     # lock the new floor after a fix lands

# Behavioral smokes (each builds what it needs; safe to run individually)
node plugins/ruflo-core/scripts/test-hooks.mjs "node $PWD/v3/@claude-flow/cli/bin/cli.js"
node plugins/ruflo-core/scripts/test-mcp-protocol.mjs
node plugins/ruflo-core/scripts/test-memory-import.mjs
node plugins/ruflo-core/scripts/test-mcp-roundtrips.mjs        # #1889 paired-tool round-trip
node plugins/ruflo-core/scripts/test-cli-no-crash.mjs          # #1863 unhandled-exception class
node plugins/ruflo-core/scripts/test-consensus-transport.mjs   # ADR-095 G2 consensus transport

# Witness manifest — regenerate + verify
node scripts/regen-witness.mjs
node plugins/ruflo-core/scripts/witness/verify.mjs --manifest verification/macos/manifest.md.json

# Temporal history
node plugins/ruflo-core/scripts/witness/history.mjs --history verification/macos/history.jsonl summary
node plugins/ruflo-core/scripts/witness/history.mjs --history verification/macos/history.jsonl regressions

Adding a new guard

1. **Behavioral smoke** → write `plugins/ruflo-core/scripts/test-<name>.mjs`. Pattern: static dist-scan first (fast, always completes), behavioral probe second with an internal timeout + a process-level watchdog so CI never hangs. Add a step to the relevant job in `v3-ci.yml`. 2. **Static gate with a baseline** → write `scripts/audit-<name>.mjs` that scans, counts violations, and fails if the count exceeds a monotone-decreasing baseline in `verification/<name>-baseline.json`. Support `--update-baseline`. Add a CI job; wire it into `witness-verify` `needs[]` if it should gate `publish`. 3. **Documented-fix marker** → append `{ id, desc, file, marker }` to `verification/witness-fixes.json`, run `node scripts/regen-witness.mjs`. The marker must be a substring the fix specifically creates (not a generic pattern like `'function'`).

Prerequisites

• Ruflo installed (`npx ruflo@latest`)
• Git repository (for rollback features)
• Node.js 18+ (for dashboard features)
• `@noble/ed25519` (for the witness verifier — a single runtime dep, `npm i @noble/ed25519`)

Quick Start

These commands ship in the current CLI:

# Change risk since HEAD~1 (without --risk it also shows classification and reviewers)
npx ruflo@latest analyze diff HEAD~1 --risk

# Complexity hot spots in a directory
npx ruflo@latest analyze complexity src/ --threshold 15

# Size, functions and TODOs per file
npx ruflo@latest analyze code -p ./src

There is no truth-scoring, `verify check` or `verify rollback` command (`ruflo verify` only checks an installed ruflo