/analyze-cluster-postmortem

description: Postmortem analysis of cluster failures → OPTIMIZE PROMPTS to work for EVERY use case
argument-hint: <cluster-id or "recent" or "dump">

**PURPOSE: Find prompt weaknesses and FIX THEM so agents work out of the box for EVERY use case.**

This is NOT about debugging infrastructure. This is about making prompts BULLETPROOF.

The Goal

Every cluster failure is a **prompt improvement opportunity**. Analyze what went wrong → Fix the prompt → Never see this failure pattern again.

Input

`$ARGUMENTS` can be:

• **Cluster ID**: Analyze specific cluster
• **"recent"**: Find most recent clusters
• **"dump"**: User will paste logs directly

Step 1: Get the Data

# List recent clusters
zeroshot list --json | jq '.[-5:]'

# Get cluster status and logs
zeroshot status $CLUSTER_ID --json
zeroshot logs $CLUSTER_ID 2>&1

# Query ledger for agent messages
sqlite3 ~/.zeroshot/clusters/$CLUSTER_ID/ledger.db "
  SELECT timestamp, topic, sender, content_text
  FROM messages
  ORDER BY timestamp ASC;
"

Step 2: Identify What Agent Did Wrong

**READ THE LOGS. What SPECIFICALLY went wrong?**

| Failure Pattern | Evidence | Prompt Gap | | ----------------------- | ------------------------------------ | ---------------------------------------- | | **Wrong files edited** | Agent edited unrelated files | Prompt doesn't scope file targets | | **Missed requirements** | Output missing key functionality | Prompt doesn't emphasize requirements | | **Broke existing code** | Tests failed after changes | Prompt doesn't enforce verification | | **Infinite loop** | Worker/validator cycle >5 iterations | Validation criteria too strict or vague | | **Wrong approach** | Used deprecated API, bad pattern | Prompt missing technical constraints | | **Incomplete work** | Partial implementation | Prompt doesn't define "done" clearly | | **Hallucinated APIs** | Called non-existent functions | Prompt doesn't ground in actual codebase | | **Ignored context** | Didn't read provided files | Context injection not working | | **Over-engineering** | Added unnecessary complexity | Prompt doesn't enforce simplicity | | **Under-testing** | No tests written | Prompt doesn't require tests |

Step 3: Trace the Prompt Chain

**Which prompt caused this behavior?**

ISSUE_OPENED (user input)
    ↓
conductor-bootstrap.json → junior-conductor/senior-conductor prompts
    ↓
CLUSTER_OPERATIONS (classification)
    ↓
{config}.json → agent prompts (worker, validator, etc.)
    ↓
Agent behavior (good or bad)

**Find the weak link:**

1. **Conductor misclassified?** → Fix `cluster-templates/conductor-bootstrap.json` 2. **Wrong config loaded?** → Fix classification logic or config selection 3. **Worker did wrong thing?** → Fix worker prompt in config template 4. **Validator too strict/loose?** → Fix validation criteria 5. **Context missing?** → Fix context injection in agent config

Step 4: Analyze Prompt Effectiveness

For Each Agent That Failed:

**1. What was the prompt?**

# Find the config that was loaded
sqlite3 ledger.db "
  SELECT content_data FROM messages
  WHERE topic = 'CLUSTER_OPERATIONS' LIMIT 1;
" | jq -r '.operations[] | select(.action == "load_config") | .config'

# Read the actual prompt from that config
cat cluster-templates/base-templates/{config}.json | jq '.agents[] | select(.id == "worker") | .prompt'

**2. What did the agent actually do?**

• Read the agent's output from logs
• Compare intended behavior vs actual behavior

**3. Where did the prompt fail to guide?**

| Prompt Issue | Symptom | Fix | | ---------------------------- | ------------------------------------ | ----------------------------------- | | **Too vague** | Agent made random choices | Add specific constraints | | **Too restrictive** | Agent couldn't solve problem | Loosen constraints, add flexibility | | **Missing edge case** | Agent broke on specific input | Add explicit handling | | **Wrong emphasis** | Agent focused on wrong thing | Reorder priorities, use CAPS | | **No verification step** | Agent declared done without checking | Add explicit verification | | **No examples** | Agent misunderstood format | Add concrete examples | | **Conflicting instructions** | Agent did inconsistent things | Remove contradictions |

Step 5: Check Validation Criteria

**Validation failures are PROMPT BUGS, not agent bugs.**

If validator rejects good work:

• Validation criteria too strict
• Validation prompt doesn't understand the task

If validator approves bad work:

• Validation criteria too loose
• Validation prompt missing checks

# Find validation results
sqlite3 ledger.db "
  SELECT sender, content_text, content_data
  FROM messages
  WHERE topic = 'VALIDATION_RESULT';
"

**Questions to answer:**

1. Did validator check the RIGHT things? 2. Did validator understand the requirements? 3. Was rejection reason valid or false positive? 4. Was approval justified or false negative?

Step 6: Generate Prompt Improvements

Report Format

# Prompt Analysis: [Cluster ID]

## Summary

- **Task**: [what user asked for]
- **Outcome**: [success/failure/partial]
- **Root Cause**: [which prompt failed and why]

---

## 🔴 FAILURE ANALYSIS

### What Went Wrong

> [Specific quote from logs showing the failure]

### Why It Went Wrong

[Analysis of which prompt instruction was missing/wrong/vague]

### The Prompt Gap

Current prompt says: "..." But agent needed: "..."

---

## 📊 AGENT BEHAVIOR AUDI