/skill-doctor

name: skill-doctor
description: 'Use when a user wants agent setup graded from conversation history. Not for skill fixing: use agent-surface-forge. Not for security scanning: use skill-scanner.'

Skill doctor

Contract

| Field | Bound contract | |---|---| | Trigger | User wants agent setup graded from conversation history. | | Authority | Reversible local: writes only to a scratch report directory under the current working tree; rollback is deleting that directory. No remote mutation. | | Side effect | Creates one scratch report directory containing report.html and supporting assets. Never modifies real skill files, configuration, or conversation history. | | Done | report.html exists in the scratch directory and renders a normalized 0-10 efficiency score, a normalized 0-10 code quality score (or N/A when the session has no code changes), per-turn findings with evidence citations (or no findings when every scored sub-score is at maximum), and ranked improvement suggestions (or none when every scored sub-score is at maximum). |

Inputs

1. **Conversation history**: a session transcript from the current agent harness. Accept one of:

• A file path to a transcript log (JSON, JSONL, or plain-text log).
• A directory of transcript files; process the most recent file.
• If neither is supplied, attempt to locate the most recent session transcript in the default harness log directory.

2. **Output directory** (optional): path for the scratch report directory. Default: `./skill-doctor-report-<timestamp>`.

Procedure

1. **Collect session data.**

• Locate the conversation history from the supplied path, directory, or default harness log.
• If no transcript is found, write a minimal report.html stating "No conversation history found" and stop.
• Read the transcript file. If the file is empty or contains fewer than 2 turns, write a minimal report.html stating "Session too short to assess" and stop.

Done when: the transcript is located and read, or a minimal report is written for the empty/short case.

2. **Decode transcript into structured turns.**

• Parse each message into a turn record with fields: `role` (user/assistant/tool), `content` (text), `tool_name` (if applicable), `tool_input` (if applicable), `tool_output` (if applicable), `timestamp` (if available).
• For JSON/JSONL transcripts, parse each line or object directly.
• For plain-text logs, identify turn boundaries by role markers (e.g., "User:", "Assistant:", "Tool:", or harness-specific prefixes). Extract tool calls from fenced code blocks or structured markers.
• Produce an ordered list of turn records.

Done when: an ordered list of turn records is produced.

3. **Score efficiency (0-10).** Evaluate the agent's task-completion efficiency against these criteria:

• Task completion (0-3): Did the agent accomplish the stated goal? 3 = fully complete, 2 = mostly complete with minor gaps, 1 = partial progress, 0 = no progress.
• Unnecessary exploration (0-3, inverted): Did the agent read or explore files unrelated to the task? 0 = extensive off-topic exploration, 1 = moderate, 2 = minimal, 3 = only task-relevant files touched.
• Retry waste (0-2, inverted): How many failed attempts or redundant retries occurred? 2 = none, 1 = one retry, 0 = multiple retries.
• Turn economy (0-2, inverted): Was the turn count reasonable for the task complexity? 2 = concise, 1 = slightly verbose, 0 = significantly excessive turns.
• Sum sub-scores for the raw efficiency score (0-10).

Done when: the efficiency score (0-10) is computed.

4. **Score code quality (0-10).** Evaluate the agent's code changes against these criteria:

• Correct file identification (0-3): Did the agent modify the right files and symbols? 3 = all correct, 2 = one wrong file, 1 = multiple wrong files, 0 = entirely wrong targets.
• Minimal diff (0-3): Were changes precise and minimal? 3 = surgical edits, 2 = minor unnecessary changes, 1 = moderate bloat, 0 = large unnecessary rewrites.
• Pattern adherence (0-2): Did changes follow existing codebase conventions? 2 = fully consistent, 1 = minor style deviations, 0 = ignores existing patterns.
• Regression avoidance (0-2): Did changes avoid introducing bugs, dead code, or broken imports? 2 = clean, 1 = minor issues, 0 = introduced regressions.
• Sum sub-scores for the raw code quality score (0-10).
• If the session contains no code changes, set code quality to N/A and state the reason in the score summary; do not emit this as a finding.

Done when: the code quality score (0-10 or N/A) is computed.

5. **Normalize scores.**

• Efficiency normalized = raw efficiency score (already 0-10).
• Code quality normalized = raw code quality score (already 0-10), or N/A if no code changes.

Done when: both scores are normalized.

6. **Compile findings with evidence.**

• For each sub-score that is not at maximum, produce a finding with:
• Category name (e.g., "Unnecessary exploration").
• Score and maximum.
• Evidence: cite specific turn numbers and content excerpts that justify the score.
• Suggestion: one concrete, actionable improvement.
• Rank suggestions by potential score impact (highest possible improvement first).

Done when: all non-maximum sub-scores have findings with evidence and ranked suggestions.

7. **Generate report.html.**

• Create the output directory.
• Write report.html as a self-contained HTML file with embedded CSS (no external dependencies, no network requests).
• Structure:
• Header: "Skill Doctor Report" with session identifier and timestamp.
• Score summary: two horizontal bar charts (efficiency, code quality) showing normalized scores out of 10. If code quality is N/A, show "N/A" instead of a bar and state the reason beside it.
• Findings section: each finding as a card with category, score, evidence block (cited turn excerpts in a styled blockquote), and suggestion.

-