/analyze

name: analyze
description: Analyze recent runs - per-model latency, tokens, and verdict agreement - and suggest model/reasoning/fanout tuning. Advisory, read-only.
allowed-tools: mcp__deliberation__analyze, Read, WebFetch
timeout: 60000

Analyze (panel performance + value)

On-demand answer to "is my model panel pulling its weight?" Reads the opt-in debug log (per-model latency, tokens, reasoning effort) and the session store (verdict agreement), then renders a human report with tuning suggestions. It writes nothing - every config change is yours to apply.

Why this exists

In a parallel fan-out (`/ask-all`), wall-time is the SLOWEST model, not the average - so one slow model that rarely says anything the others didn't sets the clock for the whole command. This surfaces those, plus error-prone and low-agreement models, from real measured data instead of guesswork.

Two lenses (never joined)

The two data stores share no run id, so they are reported side by side and never correlated by timestamp:

• **Lens A - timing/cost** (debug log): per provider+model p50/p95/max latency,

mean tokens (HTTP providers only), error rate, reasoning effort seen.

• **Lens B - agreement** (sessions): how often a model's review verdict matched

the run's final verdict. A model that is both slow (A) AND near-100% agreement (B) is the strongest cut candidate - presented as a candidate, not a fact.

Workflow

1. **Call the tool** in ONE turn:

   mcp__deliberation__analyze({})

Optional args:

• `groupBy: "config"` - compare recorded config snapshots and activations.

Narrow with full SHA-256 `configId` (or `unknown`) and optional `activationId`. Historical groups retain recorded models regardless of `configuredOnly`. Recommendations are scoped to one config and tool/expert/context cohort. Whole-run outcomes require journals; timing and agreement stay independent.

• `since` - select runs by start time, including their later terminal outcomes:

`30m`, `24h`, `7d`, or a bare number of seconds. Omit for all time. It gates BOTH lenses so timing and agreement cover the same period; an invalid value comes back as `{ error: "invalid-since" }` rather than a silent all-time report.

• `configuredOnly` - default true. Only models present in the current config are

reported; the rest are listed in `meta.excluded` with a reason. Pass `false` to include retired models.

• `sessions` (records read for Lens B; default -1 = no caller cap, bounded to 500),

`limitBytes` (debug-log tail; default 1 MB, or 32 MB with `since`).

If the result has an `error` key, report `detail` and stop - there is no analysis in that response.

2. **Handle "insufficient data".** If `meta.insufficientData` is true, the debug log is empty or off. **Check `meta.window` first**: with a window set, the honest message is "no runs in the last <since>", not "enable the debug log". Otherwise tell the user to enable it and re-run - do NOT invent numbers:

   No timing data yet. Enable it in ~/.config/deliberation/config.json:
     "debug": { "enabled": true }
   then run a few /ask-all or /consensus calls and re-run /deliberation:analyze.

(Agreement (Lens B) additionally needs `sessions.persist: true`.)

3. **Render Lens A** - a table sorted slowest-p95 first: `provider | model | calls | okCalls | p50 / p95 / max ms | mean tokens | errors | reasoning`. Add a one-line read of the slowest model and the panel's fast/slow spread.

**Latency covers SUCCESSFUL calls only** (`okCalls` is the denominator), so a timeout is an error, not a slow call. When `okCalls` is 0 the latency fields are `null` - print a dash, never 0. Say so in one line when any row has `okCalls < calls`, so the reader knows the error column and the latency column count different things.

**State the period the report covers.** With `meta.window`, lead with the window and its ACTUAL coverage (`coverageFromMs`), not the requested one: they differ when the byte tail bounded the read first. If `meta.truncated.log` or `.sessions` is true, say the data was cut short and by which bound.

**If `meta.excluded` is non-empty**, add one line naming how many models were hidden and why (grouped by `reason`). Do not analyse those rows - they are shown so the filter is visible, not so it can be second-guessed. If `meta.configError` is set, say the config could not be parsed and that the filter was skipped, and point at `/deliberation:doctor`. Surface every `meta.warnings` entry verbatim.

4. **Render Lens B** (only if `agreement` is non-empty) - a table: `provider | model | votes | agreement % | abstained`, least-agreeing first. Note that abstain-only models (ask-all runs have no verdict) carry no signal.

**When Lens B is empty, say WHY** (use the `meta` fields - do not guess):

• `sessionsPersist` is false -> persistence is off; enable `sessions.persist: true`.
• `sessionsPersist` true but `sessionsRead` is 0 -> the server read no records from

`meta.sessionsDir`. Either nothing has run yet, OR the running server resolved a different sessions dir than where records were written (an `XDG_CACHE_HOME` / `DELIBERATION_SESSIONS` drift). Print `meta.sessionsDir` and point to `/deliberation:doctor`, which compares it to the shell-resolved path.

• `sessionsRead` > 0 but `meta.agreementVotes` is 0 -> records exist but none carry a

per-opinion verdict (they are old records or `ask-all` runs, which have no verdict). Tell the user to run a fresh `/consensus` to populate Lens B - the data is not lost, it just predates verdict capture / wasn't a consensus run.

5. **Render keep/cut candidates** from `outliers` + `recommendations`. For each recommendation print its `action` and `rationale`. Separate the two targets:

• `target: "deliberation"` -> show the exact `config.json` edit

(`configKey`