/agent-wiki-tasks

name: agent-wiki-tasks
description: Discover task families across summaries and write per-family comparison pages with findings narrative. Updates wiki-twobatch/_config.yaml task definitions and writes tasks/<slug>__task.md.

Agent Wiki — Task Comparisons

Overview

Two cognitive moves in one pass:

1. **Discover** — read across all summaries and identify task families (groups of sessions that attempted the same thing across trials and conditions). 2. **Compare** — for each family, write a `tasks/<slug>__task.md` page with a per-trial table and a findings narrative that calls out the experimental signal.

This is the cross-trajectory **analysis** pass of the `agent-wiki` family.

When to run

• After enough summaries exist that a comparative pattern is visible

(typically ≥3 sessions per family).

• When the experiment design (e.g. trial × condition matrices) explicitly

cries out for a comparison page.

Workflow

Step 1: Read the corpus

uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py dump-summaries > /tmp/summaries.json

Output is a JSON array of one row per summary: `{session_id, goal, family, trial, condition, tool_calls, errors, recall_used, summary_filename}`. `family`, `trial`, `condition` come from existing classification rules — they may be null if no rule has matched yet.

Read the file:

Read /tmp/summaries.json

Step 2: Decide task families

For each candidate task family:

• **Slug**: kebab-case identifier (e.g. `extract-focal-length`).
• **Family**: short label used to group sessions (often equals slug, but

can be looser e.g. `focal-length` for a slug `extract-focal-length`).

• **Family-match rules**: how a future session gets classified. Currently

supported: `goal_substring: [list of substrings]`. A session matches the family if its `goal` contains any substring (case-insensitive).

• **Tags**: a few short tags.
• **Intro**: 1–2 sentences setting up the question.
• **Findings**: 2–5 bullets summarizing what the data shows. **This is

the actual product** — a comparison page without findings is just a table.

Rules:

1. **A family needs ≥3 sessions.** Smaller groups should not get their own page. 2. **Findings must be evidence-grounded.** Cite tool-call counts, error counts, recall-used Y/N from the dump. 3. **Don't repeat what's in the table.** Findings should explain *why* the metrics differ, not restate them. 4. **Use overrides** for sessions whose `goal` doesn't auto-match. The override key in `_config.yaml/session_family_overrides` is the session id.

Step 3: For each family, output JSON

{
  "slug": "extract-focal-length",
  "title": "Extract focal length from JPEG EXIF",
  "family": "focal-length",
  "family_match": {
    "goal_substring": ["focal length"]
  },
  "intro": "Question template: *what focal length was used to take @sample.jpg?* FocalLength (tag 0x920A) and FocalLengthIn35mmFilm (tag 0xA405) live in the Exif sub-IFD.",
  "findings": "**Net signal:** the gap between IFD0/GPS-only scripts and the Exif sub-IFD is the dominant cost. Sessions whose recall pointed at a script that already covered the sub-IFD finished in 2-3 tool calls; sessions that had to write an inline parser took 5+.",
  "tags": ["exif", "focal-length", "comparison"]
}

Pipe to:

echo '<json>' | uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py render-task

The helper:

• Updates `_config.yaml/tasks.<slug>` entry.
• Reads classified sessions; selects those matching `family`.
• Writes `tasks/<slug>__task.md` with the per-trial table + findings.

Step 4: Add overrides if needed

If a session that *should* be in a family didn't classify automatically, patch `_config.yaml`:

echo '{"session_family_overrides": {"<session-id>": {"family": "image-dims", "trial": 0, "condition": "claude_md_strong"}}}' \
  | uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py update-config

Step 5: Subtask pass — mandatory before refresh

Before refreshing indexes, scan the corpus for **subtask candidates**. The default reflex of "the dataset is uniform, no subtasks needed" is wrong for almost every dataset; even a 30-session benchmark of short workflows typically has 4-6 subtask-worthy sessions. See "## Subtasks" below for the heuristics + JSON contract + a worked example.

The minimum viable subtask layer for a condition × trial dataset: one subtask per condition, anchored in the session that best demonstrates that condition's distinctive behavior. Don't write 5 redundant subtasks when 1 representative captures the pattern.

Step 6: Refresh indexes

uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py catalog

This re-reads `_config.yaml`, re-classifies every summary, regenerates each `tasks/<slug>__task.md`, scans `tasks/<slug>__subtask.md` files, and regenerates `tasks/index.md` and the root `index.md`.

Subtasks: per-session workstream pages

The `tasks/` directory holds *two* kinds of pages distinguished by filename suffix:

• **`<slug>__task.md`** — cross-session task-comparisons (the workflow above).
• **`<slug>__subtask.md`** — narrative slices of a *single* session.

After Step 5 above, run a **second pass** to scan for subtask candidates. Don't skip this just because the dataset is uniform — a 30-session benchmark of short workflows still has 4-6 subtask-worthy sessions. The default "there are no subtasks worth writing" reflex is wrong for almost every dataset.

When to propose a subtask

Treat each session in the corpus as a potential subtask candidate. **Promote** to a subtask page when at least one of these is true:

1. **Exemplar of a condition or arc.** When the corpus has experimental conditions (`no_recall` / `guidelines` / `skill`, or arc-1 / arc-2), pick the session that best demonstrates *that condition's* distinctive behavior — its representative-best, represent