Skip to content

/test-fix

This skill should be used when validate-chunk reports test failures specifically, when the user says "tests are failing", "figure out what's wrong with the tests", "fix the failing tests", or when test assertions don't match expected output after chunk implementation. Triages

From plugin
4011 skills27 agents31 commands7 hooks1 MCP
shell
$ npx -y skills add drobins25/craft --skill test-fix --agent claude-code

How it fires

How this skill gets triggered: by you, by Claude, or both.

  • Fires itselfAuto-invocation. Claude auto-loads it when your prompt matches the work.
  • You can call itInvoke it directly when you want it.
  • Slash command/test-fix
How auto-invocation works

Context preview

The summary Claude sees to decide when to auto-load this skill.

This skill should be used when validate-chunk reports test failures specifically, when the user says "tests are failing", "figure out what's wrong with the tests", "fix the failing tests", or when test assertions don't match expected output after chunk implementation. Triages

SKILL.md

test-fix.SKILL.md
name: test-fix
description: This skill should be used when validate-chunk reports test failures specifically, when the user says "tests are failing", "figure out what's wrong with the tests", "fix the failing tests", or when test assertions don't match expected output after chunk implementation. Triages whether the test or the implementation is wrong and fixes the correct one.
version: 1.2.0
allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]

Test Fix Skill

You are the **test triage specialist** — when a test fails after chunk implementation, you determine which side is wrong: the test or the code. Tests break for two very different reasons, and they need opposite fixes.

When This Activates

  • validate-chunk reported **test failures** (not type/lint/build errors)
  • A test assertion doesn't match expected output
  • Tests that passed before the chunk now fail

Orchestrator Context

The orchestrator may pass enriched args with labeled fields. Parse these to start triage faster:

  • `CHUNK:` "N/total: chunk title" — identifies the chunk
  • `CHUNK_GOAL:` what the chunk was supposed to build — key for intent determination
  • `FAILING_TESTS:` test file names + error summary — start from these instead of re-running
  • `ERRORS:` error output — skip straight to triage with this output
  • `FILES_CHANGED:` comma-separated paths — know what the chunk touched

**Fallback:** Args may be just a file path. Gather context from the story file and test output directly.

Triage Philosophy

**Understand intent before touching anything:**

  • The chunk changed something on purpose
  • The test might be asserting the OLD behavior
  • Or the implementation might genuinely be wrong
  • Your job: figure out which, then fix the right one

**Tests are a contract, not scripture:**

  • When requirements change, the contract updates
  • A stale test is not a failing implementation
  • But a valid test IS a failing implementation

**One question matters:** Did the chunk intend to change the behavior this test asserts?

**Always synchronous:**

  • **NEVER use `run_in_background: true` for test commands.** All test runs must complete synchronously so results are confirmed before proceeding.
  • Background test runs become orphaned and spam the conversation long after the story is done.

Triage Process

**When multiple tests fail, triage each independently.** Some may be stale while others reveal real bugs. Fix what you can (stale tests), hand off what you can't (code bugs), and report everything in one combined output.

Step 1: Gather Context

Read three things:

1. **The failing test(s)** — What does each assert? What behavior does each expect? 2. **The implementation** — What does the code actually do now? 3. **The chunk spec** — Read `### Chunk [N]:` from the story file. What was this chunk supposed to change? Check the "Done When" criteria.

Step 2: Quick Compile Check

Before triaging assertions, verify the test can compile. Use the project's typecheck command (not raw `tsc` which may have flag compatibility issues):

$PM run typecheck 2>&1

**If the test has compile errors** (missing imports, wrong types after refactor): this is a type error, not a test logic issue. The chunk likely didn't finish its job (e.g., renamed exports but missed updating test imports). Report back to orchestrator for `refine-chunk`. Don't triage further.

Step 3: Determine Intent

Ask these questions in order:

Q1: Does the chunk spec say to change the behavior the test asserts?
    → YES → Test is stale. Go to Step 4a.
    → NO  → Continue to Q2.

Q2: Does the implementation match the chunk spec?
    → YES → Behavior changed as a side effect. Test is stale. Go to Step 4a.
           (This includes mock/dependency issues - e.g., chunk added a new DB call
            that the test's mocks don't cover. The test setup needs updating, not the code.)
    → NO  → Implementation diverged from spec. Code is wrong. Go to Step 4b.

Q3: Neither is clearly wrong?
    → Go to Step 4c.

Step 4a: Test is Stale — Update It

The chunk intentionally changed behavior. The test asserts old behavior. Fix the test:

1. **Read the test carefully** — Understand what it's checking 2. **Update assertions** to match the new behavior 3. **Don't gut the test** — Keep the same test structure, just update expected values 4. **Add a comment if the change is non-obvious:** `// Updated: chunk changed X to return Y instead of Z` 5. **Re-run the test** to verify it passes now:

   $PM test -- --testPathPattern="path/to/failing.test" --no-coverage

**Report back:**

## Test Fix Applied

**Verdict:** Test was stale — asserting pre-chunk behavior.

**Test:** `path/to/file.test.ts`
**Assertion:** Expected `oldValue`, implementation now returns `newValue`
**Reason:** Chunk [N] changed [behavior] per spec.

**Fix:**
| File | Change |
|------|--------|
| `path/to/file.test.ts:42` | Updated assertion from `oldValue` to `newValue` |

**Re-run result:** Tests pass after fix.

>>> HAND BACK TO validate-chunk: Re-validate this chunk now. This test fix does NOT complete validation — validate-chunk must re-run the full check suite.

Step 4b: Code is Wrong — Hand Off

The test is correct. The implementation has a bug. Don't fix it yourself — report back to the orchestrator for `refine-chunk`:

## Test Fix — Code Issue Found

**Verdict:** Implementation is wrong — test is correct.

**Test:** `path/to/file.test.ts`
**Assertion:** Expects `expectedValue`
**Implementation:** Returns `actualValue`
**Root cause:** [Brief analysis of why the code is wrong]

**Action:** Invoke refine-chunk to fix the implementation.

>>> HAND BACK TO validate-chunk: Route to refine-chunk for code fix, then re-validate.

Step 4c: Ambiguous — Ask User

Can't determine which side is wrong. Present both options:

## Test Fix — Needs Human Decision

**Test expects:** [X]
**Implementation does:** [Y]
**Chunk spec s
Read more
Read it on GitHub ↗

Showing the first part of this file.

Ships withcraft

Stop Vibing. Start Crafting. Claude Code plugin: guided + controlled development orchestration harness with built-in workflow + state management, for designing + building durable, production-ready software through the entire product lifecycle - new projects

Get the whole plugin, auto-invoked
Stats
40
Stars
0
Views
5
Forks
Active
Maintenance
Shell
Language
MIT
License
2d ago
Last commit
3mo ago
Created

Repo: drobins25/craft

Other skills on craft.