/judge-with-debate

name: judge-with-debate
description: Evaluate solutions through multi-round debate between independent judges until consensus
argument-hint: Solution path(s) and evaluation criteria

judge-with-debate

<task> Evaluate solutions through multi-agent debate where independent judges analyze, challenge each other's assessments, and iteratively refine their evaluations until reaching consensus or maximum rounds. </task>

<context> This command implements the Multi-Agent Debate pattern for high-quality evaluation where multiple perspectives and rigorous argumentation improve assessment accuracy. Unlike single-pass evaluation, debate forces judges to defend their positions with evidence and consider counter-arguments.

Key benefits:

• **Structured evaluation** - Meta-judge produces tailored rubrics and criteria before judging begins
• **Multiple perspectives** - Three independent judges reduce individual bias
• **Evidence-based debate** - Judges defend positions with specific evidence from the solution and evaluation specification
• **Iterative refinement** - Up to 3 debate rounds drive convergence on accurate scores
• **Shared specification** - Meta-judge runs once; all judges across all rounds share the same evaluation specification

</context>

Pattern: Debate-Based Evaluation

This command implements iterative multi-judge debate:

Phase 0: Setup
         mkdir -p .specs/reports
                  |
Phase 0.5: Dispatch Meta-Judge
         Meta-Judge (Opus)
              |
         Evaluation Specification YAML
              |
Phase 1: Independent Analysis (3 judges in parallel)
         +- Judge 1 -> {name}.1.md -+
Solution +- Judge 2 -> {name}.2.md -+-+
         +- Judge 3 -> {name}.3.md -+ |
                                      |
Phase 2: Debate Round (iterative)     |
    Each judge reads others' reports  |
         |                            |
    Argue + Defend + Challenge        |
    (grounded in eval specification)  |
         |                            |
    Revise if convinced --------------+
         |                            |
    Check consensus                   |
         +- Yes -> Final Report       |
         +- No -> Next Round ---------+

Process

Setup: Create Reports Directory

Before starting evaluation, ensure the reports directory exists:

mkdir -p .specs/reports

**Report naming convention:** `.specs/reports/{solution-name}-{YYYY-MM-DD}.[1|2|3].md`

Where:

• `{solution-name}` - Derived from solution filename (e.g., `users-api` from `src/api/users.ts`)
• `{YYYY-MM-DD}` - Current date
• `[1|2|3]` - Judge number

Phase 0.5: Dispatch Meta-Judge

Before independent analysis, dispatch a meta-judge agent to generate a tailored evaluation specification. The meta-judge runs ONCE and produces rubrics, checklists, and scoring criteria that ALL judges will use across ALL rounds.

**Meta-judge prompt template:**

## Task

Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that multiple judge agents will use to evaluate the solution through independent analysis and multi-round debate.

CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`

## User Prompt
{task description - what the solution was supposed to accomplish}

## Context
{Any relevant context about the solution being evaluated}

## Artifact Type
{code | documentation | configuration | etc.}

## Evaluation Mode
Multi-judge debate with consensus-seeking across rounds

## Instructions
Return only the final evaluation specification YAML in your response.
The specification should support both independent analysis and debate-based refinement.

**Dispatch:**

Use Task tool:
  - description: "Meta-judge: generate evaluation specification for {solution-name}"
  - prompt: {meta-judge prompt}
  - model: opus
  - subagent_type: "sadd:meta-judge"

Wait for the meta-judge to complete and extract the evaluation specification YAML from its output before proceeding to Phase 1.

Phase 1: Independent Analysis

Launch **3 independent judge agents in parallel** (Opus for rigor):

1. Each judge receives:

• Path to solution(s) being evaluated
• The meta-judge's evaluation specification YAML
• Task description

2. Each produces **independent assessment** saved to `.specs/reports/{solution-name}-{date}.[1|2|3].md` 3. Reports must include:

• Per-criterion scores with evidence
• Specific quotes/examples supporting ratings
• Overall weighted score
• Key strengths and weaknesses

**Key principle:** Independence in initial analysis prevents groupthink.

**Prompt template for initial judges:**

You are Judge {N} evaluating a solution independently against an evaluation specification produced by the meta judge.

CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`

## Solution
{path to solution file(s)}

## Task Description
{what the solution was supposed to accomplish}

## Evaluation Specification

```yaml
{meta-judge's evaluation specification YAML}

Output File

.specs/reports/{solution-name}-{date}.{N}.md

Instructions

Follow your full judge process as defined in your agent instructions!

Additional instructions: 1. Read the solution thoroughly 2. For each criterion from the evaluation specification:

• Find specific evidence (quote exact text)
• Score on the defined scale
• Justify with concrete examples

3. Calculate weighted overall score 4. Write comprehensive report to {output_file}

Add to report beginning `Done by Judge {N}`

**Dispatch each judge:**

Use Task tool:

• description: "Judge {N}: independent analysis of {solution-name}"
• prompt: {judge prompt with evaluation specification YAML}
• model: opus
• subagent_type: "sadd:judge"

### Phase 2: Debate Rounds (Iterative)

For each debate round (max 3 rounds):

Launch **3 debate agents in parallel**:

1. Each judge agent receives:
   - Path to their own previous report (`.specs/repo