/printing-press-output-review

name: printing-press-output-review
description: >
  Internal sub-skill: agentic review of a printed CLI's sampled command output for
  plausibility issues that rule-based checks can't encode (substring-match
  relevance, format bugs, silent source drops, ranking failures). Invoked via the
  Skill tool by the main printing-press skill at Phase 4.85 and printing-press-polish
  SKILL.md during the diagnostic loop. Not for direct user invocation — its
  actionable wrappers are /printing-press and /printing-press-polish.
context: fork
user-invocable: false
allowed-tools:
  - Bash
  - Agent
created_by: user

printing-press-output-review (internal)

Review the sampled outputs from a printed CLI for plausibility bugs that dogfood, verify, and the rule-based `scorecard --live-check` rules can't catch. Wave B policy: all findings surface as warnings, never errors.

This skill is **internal-only** (`user-invocable: false`). It's invoked by parents — the main printing-press skill at its shipcheck Phase 4.85, and the printing-press-polish skill during its diagnostic loop. Running it standalone would produce floating findings text with no ship verdict, no fixes applied, no publish offer; the actionable wrappers are `/printing-press` and `/printing-press-polish`. The skill carries `context: fork` so the reviewer agent's diagnostic chatter stays isolated from the calling skill's context.

Input

The caller passes `$CLI_DIR` as the argument: an absolute path to the printed CLI's working directory.

What this catches

Bugs that rule-based checks miss, typically surfaced by 5 minutes of hands-on testing but slipping past dogfood, verify, and `scorecard --live-check` rules:

• Substring-match results that coincidentally contain the query but don't match semantically (e.g., a query matches a substring of a larger unrelated term)
• Aggregation commands silently dropping sources when only some of the requested N come back
• Ranking or sort commands returning top-N results that aren't plausibly the best for the query (broken weights, extractor fallbacks)
• URLs in output pointing at category index pages, feed endpoints, or random-selector routes rather than canonical content permalinks
• Format bugs the rule-based layer doesn't catch (mojibake, inconsistent pluralization, truncated/wrapped cell content)

Procedure

Step 1: Gather sample data

# Locate research.json. Adjacent to the binary covers the post-promote
# layout (standalone polish, shipcheck against the library copy). The
# grandparent fallback covers mid-pipeline invocations where $CLI_DIR is
# $PRESS_RUNSTATE/runs/<id>/working/<cli> and research.json lives at
# $PRESS_RUNSTATE/runs/<id>/research.json. Without the fallback, scorecard
# reports `unable: true` mid-pipeline and we SKIP the most informative review.
# Use a bash array so the flag survives paths with spaces.
RESEARCH_ARGS=()
if [ ! -f "$CLI_DIR/research.json" ]; then
  _grandparent="$(dirname "$(dirname "$CLI_DIR")")"
  if [ -f "$_grandparent/research.json" ]; then
    RESEARCH_ARGS=(--research-dir "$_grandparent")
  fi
fi

cli-printing-press scorecard --dir "$CLI_DIR" "${RESEARCH_ARGS[@]}" --live-check --json > /tmp/output-review-livecheck.json 2>&1 || true

If the scorecard call fails or `/tmp/output-review-livecheck.json` is empty, return the SKIP result (Step 3) without dispatching the reviewer.

Before dispatch, count entries in `live_check.features[]` whose `status` is `pass`. If there are zero, return `SKIP` with the reason `no eligible passing samples; plausibility not assessed`. Do not dispatch the reviewer, and never report a clean `PASS` merely because all sampled commands failed or were excluded from review.

Step 2: Dispatch the reviewer agent

Use the Agent tool (general-purpose) with this prompt contract:

> Review the sampled outputs from the shipped CLI at `$CLI_DIR`. You have these ground-truth sources: > > - Sampled command output: read `/tmp/output-review-livecheck.json` and inspect the `live_check.features[]` array. Each entry has the command, example invocation, redacted stdout evidence (in `output_sample`, bounded to ~4 KiB), the redacted pass/fail reason, and a `warnings` array (populated by rule-based checks like the raw-HTML-entity detector). Treat `<redacted>` markers as privacy scrubbed values, not format bugs. > - **Review only `status: pass` entries.** Entries with `status: fail` either crashed, timed out, or had placeholder args (`<id>`, `<url>`) that never produced real output — their sample is empty and there's nothing for you to judge. > - `$CLI_DIR/research.json` `novel_features` (planned behavior per feature) and `novel_features_built` (verified built commands). > - The CLI binary at `$CLI_DIR/<cli-name>-pp-cli` — you may invoke additional commands to gather more output when a finding needs verification. > > For each of these checks, report findings under 50 words each. Only report issues a human user would notice in 5 minutes of hands-on testing — not every edge case a thorough QA pass might find: > > 1. **Output *semantically* matches query intent.** For sampled novel features with a query argument, judge relevance beyond what the mechanical query-token check in live-check already enforced. A feature that passed live-check's `outputMentionsQuery` test still contains *some* query token somewhere — but "buttermilk" appearing as a substring of "butter" results, or "brownies" returning a chili recipe because the extractor fell back to adjacent content, both slip past the mechanical check. Only flag when a human user would look at the top results and say "this isn't what I asked for." Skip this check when the example has no query argument. > 2. **No obvious format bugs.** Does the output contain raw HTML entities, mojibake (question marks or replacement chars in titles), or malformed URLs (pointing at category index pages, feed endpoints, or random-selector routes rather than canonical content permalinks)? Rule-based li