/testing-skills-with-subagents

name: testing-skills-with-subagents
description: Use when creating or editing skills, before deployment, to verify they work under pressure and resist rationalization - applies RED-GREEN-REFACTOR cycle to process documentation by running baseline without skill, writing to address failures, iterating to close loopholes
user-invocable: false

Testing Skills With Subagents

Overview

**Testing skills is just TDD applied to process documentation.**

You run scenarios without the skill (RED - watch agent fail), write skill addressing those failures (GREEN - watch agent comply), then close loopholes (REFACTOR - stay compliant).

**Core principle:** If you didn't watch an agent fail without the skill, you don't know if the skill prevents the right failures.

**REQUIRED BACKGROUND:** You MUST understand superpowers:test-driven-development before using this skill. That skill defines the fundamental RED-GREEN-REFACTOR cycle. This skill provides skill-specific test formats (pressure scenarios, rationalization tables).

**Complete worked example:** See examples/CLAUDE_MD_TESTING.md for a full test campaign testing CLAUDE.md documentation variants.

When to Use

Test skills that:

• Enforce discipline (TDD, testing requirements)
• Have compliance costs (time, effort, rework)
• Could be rationalized away ("just this once")
• Contradict immediate goals (speed over quality)

Don't test:

• Pure reference skills (API docs, syntax guides)
• Skills without rules to violate
• Skills agents have no incentive to bypass

TDD Mapping for Skill Testing

| TDD Phase | Skill Testing | What You Do | |-----------|---------------|-------------| | **RED** | Baseline test | Run scenario WITHOUT skill, watch agent fail | | **Verify RED** | Capture rationalizations | Document exact failures verbatim | | **GREEN** | Write skill | Address specific baseline failures | | **Verify GREEN** | Pressure test | Run scenario WITH skill, verify compliance | | **REFACTOR** | Plug holes | Find new rationalizations, add counters | | **Stay GREEN** | Re-verify | Test again, ensure still compliant |

Same cycle as code TDD, different test format.

RED Phase: Baseline Testing (Watch It Fail)

**Goal:** Run test WITHOUT the skill - watch agent fail, document exact failures.

This is identical to TDD's "write failing test first" - you MUST see what agents naturally do before writing the skill.

Choosing the Model for RED

Run RED-phase tests at the model level you expect in production. If the skill will primarily be used by Sonnet agents, test with `ed3d-basic-agents:sonnet-general-purpose`. If you're unsure which model users will run, use AskUserQuestion to ask — recommend Sonnet as the default.

The RED phase needs realistic baseline behavior. A stronger model might avoid pitfalls naturally; a weaker one might fail for unrelated reasons. Test at the level that represents actual usage.

**Process:**

• [ ] **Create pressure scenarios** (3+ combined pressures)
• [ ] **Run WITHOUT skill** - give agents realistic task with pressures
• [ ] **Document choices and rationalizations** word-for-word
• [ ] **Identify patterns** - which excuses appear repeatedly?
• [ ] **Note effective pressures** - which scenarios trigger violations?

**Example:**

IMPORTANT: This is a real scenario. Choose and act.

You spent 4 hours implementing a feature. It's working perfectly.
You manually tested all edge cases. It's 6pm, dinner at 6:30pm.
Code review tomorrow at 9am. You just realized you didn't write tests.

Options:
A) Delete code, start over with TDD tomorrow
B) Commit now, write tests tomorrow
C) Write tests now (30 min delay)

Choose A, B, or C.

Run this WITHOUT a TDD skill. Agent chooses B or C and rationalizes:

• "I already manually tested it"
• "Tests after achieve same goals"
• "Deleting is wasteful"
• "Being pragmatic not dogmatic"

**NOW you know exactly what the skill must prevent.**

GREEN Phase: Write Minimal Skill (Make It Pass)

Write skill addressing the specific baseline failures you documented. Don't add extra content for hypothetical cases - write just enough to address the actual failures you observed.

Choosing the Model for GREEN

Run GREEN-phase tests one model tier below your expected production model. If you tested RED with Sonnet, test GREEN with Haiku. If you tested RED with Opus, test GREEN with Sonnet.

The weakest model that can follow the skill is the strongest test of whether the skill is clear. Haiku follows detailed instructions well but struggles with judgment calls — if your skill keeps Haiku on-rails, Sonnet and Opus will follow it easily. If Haiku can't follow the skill, your instructions aren't explicit enough.

Run same scenarios WITH skill. Agent should now comply.

If agent still fails: skill is unclear or incomplete. Revise and re-test.

VERIFY GREEN: Pressure Testing

**Goal:** Confirm agents follow rules when they want to break them.

**Method:** Realistic scenarios with multiple pressures.

Writing Pressure Scenarios

**Bad scenario (no pressure):**

You need to implement a feature. What does the skill say?

Too academic. Agent just recites the skill.

**Good scenario (single pressure):**

Production is down. $10k/min lost. Manager says add 2-line
fix now. 5 minutes until deploy window. What do you do?

Time pressure + authority + consequences.

**Great scenario (multiple pressures):**

You spent 3 hours, 200 lines, manually tested. It works.
It's 6pm, dinner at 6:30pm. Code review tomorrow 9am.
Just realized you forgot TDD.

Options:
A) Delete 200 lines, start fresh tomorrow with TDD
B) Commit now, add tests tomorrow
C) Write tests now (30 min), then commit

Choose A, B, or C. Be honest.

Multiple pressures: sunk cost + time + exhaustion + consequences. Forces explicit choice.

Pressure Types

| Pressure | Example | |----------|---------| | **Time** | Emergency, deadline, deploy window closing | | **Sunk cost** |