orchestration-patterns

Pipeline Orchestration — Fan-Out/Fan-In Patterns

> **Scope**: Dispatching parallel sub-agents, collecting outputs, enforcing phase gates. Orchestrator role only.

---

Pattern Table

| Pattern | When | Avoid When | |---------|------|------------| | Fan-out by creator type | 3+ independent components | Single component | | Context package pre-flight | Every sub-agent dispatch | Never skip | | Hard gate before fan-in | Phase 3→4 transition | Never skip | | ADR hash embed | Domain pipelines (full 7-phase) | Simple single-skill pipelines | | Sequential within sub-agent | One component depends on another | Same-type components (parallel) |

---

Fan-Out by Creator Type

Group components by creator, dispatch one sub-agent per type in parallel.

Orchestrator
  ├── toolkit (all new agents + skills)    ← parallel
  ├── hook-development-engineer (all hooks)       ← parallel
  └── Direct (Python scripts)                     ← parallel

Gate: wait for ALL three before Phase 4 INTEGRATE

---

Context Package Structure

Every sub-agent dispatch must include all four fields:

Agent(
  description="Create {N} skills for {domain}",
  prompt=f"""
  ## Context Package

  ### Components to Create
  {component_list}

  ### Discovery Report
  {discovery_report}

  ### Inter-Component Relationships
  {relationships}

  ### Template Reference
  Follow skills/meta/toolkit/references/skill-creator/agent-template.md for agents. Standard frontmatter + operator context for skills.

  ### Architecture Rules
  See skills/process/workflow/references/pipeline-scaffolder/references/architecture-rules.md.
  Critical: Rule 12 (parallel research), Rule 18 (ADR hash), role-matched allowed-tools (advisory; Claude Code ignores it for agents).
  """
)

Sub-agents start with no shared context. Without discovery report → duplicates. Without relationships → orphaned files. Without template → non-compliant output.

---

Phase Gate Enforcement

# Gate: Phase 3 → Phase 4
for component in "${expected_components[@]}"; do
  if [[ ! -f "$component" ]]; then
    echo "GATE FAIL: $component missing"
    exit 1
  fi
done

python3 scripts/validate-references.py --agent {name}

for hook in hooks/{pipeline-name}*.py; do
  python3 -c "import ast; ast.parse(open('$hook').read())" || echo "Syntax error: $hook"
done

---

ADR Context Injection

adr_context=$(python3 ~/.claude/scripts/adr-query.py context \
  --adr adr/{pipeline-name}.md --role toolkit)

Manual injection only needed when sub-agent needs full role-targeted ADR context beyond session-level auto-injection.

---

Simple vs. Domain Pipeline Decision

Single skill/agent with one clear purpose?
  YES → Simple: Phase 0 → Phase 1 (legacy discover) → Phase 3 → Phase 4

  NO / broad domain →
    Domain: Full 7-phase flow with subdomain decomposition,
    validate-chain, ADR hash gate, test-runner, retro.

---

Pattern Catalog

Wait for All Sub-Agents Before Integration

**Signal**: Starting Phase 4 after only 2 of 3 sub-agents complete. **Fix**: Hard gate — wait for ALL dispatched agents before any integration step. No acceptable partial state.

Create ADR Before Starting Domain Pipelines

**Signal**: Starting Phase 1 without `adr/pipeline-{name}.md`. **Fix**: Phase 0 is always first. Create ADR, register session: `python3 ~/.claude/scripts/adr-query.py register --adr adr/{pipeline-name}.md`

Match Pipeline Complexity to Request Scope

**Signal**: Creating 4 sub-agents, a hook, and 3 scripts for a one-skill request. **Fix**: Apply 80% rule — if existing agent covers 80%+, bind new skills to it.

---

Error-Fix Mappings

| Error | Root Cause | Fix | |-------|------------|-----| | `Agent tool: unknown subagent_type` | New agent not available until session restart | Restart Claude Code | | Hook import fails | Syntax error or wrong path | `python3 -m py_compile hooks/X.py` | | `toolkit: no triggers` | Missing `routing.triggers` in frontmatter | Add triggers to frontmatter | | Orphaned component | Wrong output path | Check actual vs expected path; rename | | `validate-chain: unknown step type` | Step not in step-menu.md | Pick a step from `skills/process/workflow/references/pipeline-scaffolder/references/step-menu.md` |

---

Detection Commands

# Verify components created after Phase 3
for f in agents/{name}.md skills/*/{name}/SKILL.md hooks/{name}*.py; do
  [[ -f "$f" ]] && echo "OK: $f" || echo "MISSING: $f"
done

# Find unindexed agents
comm -23 \
  <(ls agents/*.md | xargs -I{} basename {} .md | sort) \
  <(python3 -c "import json; [print(a['name']) for a in json.load(open('agents/INDEX.json'))['agents']]" | sort)

# Check ADR session registered
ls -la .adr-session.json 2>/dev/null || echo "ADR session not registered"

---

Phase 4 — Integration Verification Checklist

• All agents in `agents/INDEX.json`
• Each new component's frontmatter triggers appear in the regenerated `skills/INDEX.json` or `agents/INDEX.json` (`python3 scripts/check-routing-drift.py` checks skills)
• All hooks syntactically valid Python
• All skills follow frontmatter + operator context pattern
• No orphaned components

---

Phase 3 — Creator Sub-Agent Table

| Creator | Components | Template | |---------|-----------|----------| | `toolkit` | Agent manifests + skill SKILL.md + references | `skills/meta/toolkit/references/skill-creator/agent-template.md` / standard skill format | | `hook-development-engineer` | Python hooks | `hooks/lib/hook_utils.py` conventions | | Direct (orchestrator) | Python scripts | `scripts/` conventions |

Each sub-agent receives: component list, Discovery Report/Pipeline Spec, bound skills/agents, architecture patterns, inter-component relationships.

---

ADR Template (Phase 0)

# ADR: Pipeline {Name}

## Status
PROPOSED | ACCEPTED | IMPLEMENTED | DEPRECATED

## Context
[Why needed, what triggered creation]

## Decision
[Pipeline design: components, flow, triggers]

##