/audit-agents

description: Audit agent outputs from an orchestration run — timeline analysis, stall detection, active vs idle time breakdown
argument-hint: "[session-id | latest | <agent-id>]"
install: extras

<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios --> <!-- SPDX-License-Identifier: AGPL-3.0-or-later -->

/audit-agents — Agent Output Auditor

> **Skill names**: `{FORGE_SKILL_PREFIX}` is `forgedock:` (plugin install) or empty (`install.sh`), resolved once per run by `commands/work-on.md` § Skill Name Resolution. If the skill is not found under either name, STOP and report "skill not found" — never run the phase inline.

**Input**: $ARGUMENTS

Purpose

Parse agent JSONL output files from orchestration runs to produce actionable diagnostics:

• Per-agent phase timeline with durations
• Stall detection (gaps between phases caused by agent `end_turn` stops)
• Active vs idle time breakdown
• Resume cycle counting
• Wave-level efficiency metrics

---

Config Preamble

Before executing any phase, read `forge.yaml` to resolve project references:

CONFIG_FILE="${FORGE_CONFIG:-forge.yaml}"
if [ -f "$CONFIG_FILE" ]; then
  GH_OWNER=$(yq '.project.owner' "$CONFIG_FILE")
  GH_REPO_NAME=$(yq '.project.repo' "$CONFIG_FILE")
  GH_REPO="${GH_OWNER}/${GH_REPO_NAME}"
  # FORGE_REPO: the self-pipeline repo where orchestration metrics are tracked.
  # Set project.forge_repo in forge.yaml if your pipeline repo differs from GH_REPO.
  FORGE_REPO=$(yq '.project.forge_repo // ""' "$CONFIG_FILE")
  [ -z "$FORGE_REPO" ] && FORGE_REPO="$GH_REPO"
else
  echo "WARNING: forge.yaml not found — commands will use placeholder values"
  echo "Run: cp forge.yaml.example forge.yaml  and fill in your project details"
  GH_REPO="your-org/your-repo"
  FORGE_REPO="$GH_REPO"
fi

---

Phase 1: Locate Agent Outputs

Step 1A: Find the session

Agent outputs live in `~/.claude/projects/{munged-project-path}/subagents/agent-{agentId}.jsonl`. The project path munging rule: both `/` and `.` are replaced with `-` (e.g. `/home/user/projects/myproject` → `-home-user-projects-myproject`).

import os, sys
from pathlib import Path

def _session_project_dir(project_path):
    """Convert a Claude Code project path to its ~/.claude/projects/ directory name.
    Rule: both '/' and '.' are replaced with '-'.
    Matches pipeline-health.md _session_project_dir() exactly.
    """
    import re
    return re.sub(r'[/.]', '-', project_path)

CLAUDE_PROJECTS_DIR = Path.home() / ".claude" / "projects"

# Discover all subagent JSONL files under ~/.claude/projects/
# Each subagent file: ~/.claude/projects/{project}/{sessionId}/subagents/agent-{agentId}.jsonl
subagent_files = []
if CLAUDE_PROJECTS_DIR.exists():
    for jsonl_path in sorted(CLAUDE_PROJECTS_DIR.rglob("subagents/agent-*.jsonl"),
                             key=lambda p: p.stat().st_mtime, reverse=True):
        # Only process files with actual content (> 10 lines / non-empty)
        try:
            size = jsonl_path.stat().st_size
            if size < 200:  # 200 bytes ≈ fewer than 10 minimal JSONL lines
                continue
            # Count lines portably without wc -l
            with open(jsonl_path, 'rb') as fh:
                line_count = sum(1 for _ in fh)
            if line_count > 10:
                subagent_files.append(jsonl_path)
        except OSError:
            continue

if not subagent_files:
    print("SKIPPED — transcripts unreadable on this platform")
    print("  ~/.claude/projects/ not found or contains no subagent JSONL files.")
    sys.exit(0)

print(f"Found {len(subagent_files)} subagent JSONL file(s) under {CLAUDE_PROJECTS_DIR}")

**Input resolution:**

• `latest` or no argument → most recent session with agent outputs
• A session UUID → that specific session
• An agent ID (starts with `a`, 17+ hex chars) → find the session containing that agent
• A project path fragment (e.g., `my-project`) → filter to sessions for that project

Step 1B: Collect agent JSONL files

import os
from pathlib import Path

# Filter subagent_files by $ARGUMENTS (session UUID, agent ID, project fragment, or "latest")
ARGUMENT = "$ARGUMENTS".strip()

if not ARGUMENT or ARGUMENT.lower() == "latest":
    # Use the most recent session — take the most recently modified subagent directory
    # Group by parent session dir (grandparent of the subagent file)
    by_session = {}
    for p in subagent_files:
        session_dir = p.parent.parent  # .../projects/{project}/{sessionId}/
        mtime = session_dir.stat().st_mtime if session_dir.exists() else 0
        if session_dir not in by_session or mtime > by_session[session_dir][0]:
            by_session[session_dir] = (mtime, [])
        by_session[session_dir][1].append(p)
    if by_session:
        latest_session_dir = max(by_session, key=lambda d: by_session[d][0])
        agent_files = by_session[latest_session_dir][1]
    else:
        agent_files = []
else:
    # Filter by session UUID, agent ID, or project path fragment
    agent_files = [
        p for p in subagent_files
        if ARGUMENT in str(p)
    ]

if not agent_files:
    print(f"SKIPPED — no matching agent JSONL files found for argument: {ARGUMENT!r}")
    import sys; sys.exit(0)

print(f"Processing {len(agent_files)} agent file(s)")
for p in agent_files:
    agent_id = p.stem.replace("agent-", "", 1)
    print(f"  {agent_id}  {p}")

Only process files with `> 10` lines (smaller files are helper/polling agents, not work-on agents). Line count is computed in Python from the actual JSONL bytes — not from a symlink stub.

---

Phase 2: Parse Each Agent

For each agent JSONL file, use a Python script to extract the timeline.

**IMPORTANT**: Run this as a SINGLE Python script, not per-agent bash loops. The JSONL files can be large (500+ lines, 100KB+).

import json, sys, os
from datetime import datetime
from collections import defaultdict

def parse_agent(filepath, agent_id):
    """Par