/skill-distiller

name: skill-distiller
description: 'Converts Opus-quality skills into deterministic Haiku-executable workflows via trace-driven distillation and cross-model validation. Triggers on: "distill this skill", "make this skill work on Haiku", "cross-model optimization", "optimize skill for cost". NOT for code simplification, use code-refiner.'
metadata:
  version: 1.0.1
  category: development
  tags: [distillation, cross-model, optimization, haiku, deterministic]
  difficulty: advanced
  phase: build

Skill Distiller

Transform skills authored for high-capability models (Opus) into deterministic workflows that execute reliably on lower-cost models (Sonnet, Haiku). The core insight from EvoSkills: skills encode reusable task structure, not model-specific artifacts. A skill evolved on Opus transfers with +35-45pp gains to other models — but only when the instructions are sufficiently deterministic that lower-capability models can follow them without improvising.

Reference Files

| File | Contents | Load When | | -------------------------------------- | -------------------------------------------------- | ------------------------------------ | | `references/distillation-patterns.md` | Pattern catalog for converting reasoning to rules | Always |

Prerequisites

• The source skill must exist and pass `package-evaluator` at >= 70%
• Access to both the source model (Opus) and target model (Haiku/Sonnet) for validation
• The `surrogate-verifier` skill for cross-model assertion checking

Workflow

Phase 1: Complexity Analysis

Score each section of the source SKILL.md for reasoning difficulty:

| Complexity Signal | Score | Distillation Action | | ------------------------------------- | ----- | -------------------------------------------- | | Decision tree with 3+ branches | HIGH | Convert to explicit if/then lookup table | | "Use judgment" or "consider context" | HIGH | Replace with concrete heuristic rules | | Multi-step inference chain | HIGH | Break into numbered atomic steps | | Reference to domain expertise | MED | Add explicit reference file with knowledge | | Clear enumerated steps | LOW | Keep as-is | | Concrete examples with expected output| LOW | Keep as-is |

Produce a complexity map: section name -> complexity score -> planned action.

Phase 2: Trace Collection

Execute the source skill with Opus on 5 representative tasks:

1. Select tasks from `evals/cases.yaml` (positive cases) or generate new ones 2. For each task, capture the full execution trace:

• Tool calls made (which tools, in what order)
• Intermediate reasoning visible in output
• Final output structure and content
• Time taken and token usage

3. Store traces as structured data for pattern extraction

Phase 3: Pattern Extraction

From the collected traces, extract deterministic patterns:

1. **Decision paths** — For each HIGH-complexity section, find the actual decisions Opus made across the 5 tasks. If Opus chose the same path in 4/5 cases, that path becomes the default rule 2. **Lookup tables** — Where Opus applied domain knowledge, build explicit lookup tables (e.g., "if input contains SQL, use these patterns; if input contains Python, use those") 3. **Concrete examples** — Extract representative input/output pairs from traces to serve as few-shot examples in the distilled skill 4. **Tool sequences** — Identify the common tool invocation pattern and make it explicit ("Step 1: Read the file. Step 2: Grep for pattern X. Step 3: Write output.")

Phase 4: Distilled Rewrite

Rewrite the SKILL.md applying all distillation actions from Phase 1:

| Source Pattern | Distilled Replacement | | -------------------------------------- | ------------------------------------------------------------ | | "Analyze the code and determine..." | "Check for these 5 specific patterns: [list]" | | "Use appropriate formatting" | "Output as a markdown table with columns: [A, B, C]" | | "Consider the context to decide..." | "If [condition A]: do X. If [condition B]: do Y. Default: Z" | | "Apply best practices for..." | Reference file with explicit best practices enumerated | | Multi-paragraph reasoning instruction | Numbered step list with single-sentence steps |

Rules for the rewrite:

• Every instruction must be actionable by a model with no domain expertise
• No step should require inference — each step's input and output must be explicit
• Replace all "consider", "analyze", "determine" verbs with "check", "count", "list", "output"
• Add concrete examples for any step that could be ambiguous
• Keep the SKILL.md under 500 lines (distillation should reduce, not expand)

Phase 5: Target Model Validation

Run the distilled skill on the target model (Haiku or Sonnet):

1. Execute the same 5 tasks from Phase 2 with the distilled skill loaded 2. Use the `surrogate-verifier` to generate assertions for each task output 3. Compare pass rates:

| Metric | Source (Opus + original) | Target (Haiku + distilled) | Delta | | ------------------------------- | ------------------------ | -------------------------- | ----- | | Assertions passed | N/M | N/M | ± | | Weighted score | X.XX | X.XX | ± | | Output completeness | % | % | ± | | Format compliance | % | % | ± |

4. If target mod