/agent-session-format-check

name: agent-session-format-check
description: Verify agent session format compatibility for Agent Sessions. Use when any agent CLI updates, when monitoring flags drift, or when bumping max verified versions (fixtures + docs + tests). Covers session schema, usage/limits tracking, storage backends, and discovery path contracts for all supported agents.

Agent Session Format Check

Answer one precise question for every supported agent:

> Can current Agent Sessions code support the latest available session/storage/usage > format from the latest available agent build?

The answer must be layered. Do not collapse version checks, stale samples, schema fingerprints, discovery contracts, and usage probes into a vague severity label.

**Evidence-first:**

• Gather a report + sample paths first.
• Do not change parsers/fixtures/docs without explicit user approval.

**Related skill:** `agent-support-matrix` — maintains the matrix YAML, ledger, and update-checklist workflow. This skill focuses on *detection and evidence collection*; `agent-support-matrix` focuses on *recording and gating version bumps*.

**Process doc:** `docs/agent-support/monitoring.md` — defines the compatibility verdicts, legacy severity model, cadence, and escalation workflow that feed into this skill.

---

1 Quick Start (all agents)

1. Run weekly monitoring:

   ./scripts/agent_watch.py --mode weekly

Report path prints to stdout and is written under `scripts/probe_scan_output/agent_watch/*/report.json`.

2. In `report.json`, check each agent under `results.<agent>`:

• `compatibility.verdict`, `compatibility.scope`, `compatibility.blockers`,

and `compatibility.next_action`

• `verified_version`, `installed.parsed_version`, `upstream.parsed_version`
• `compatibility.latest_status` to distinguish `current_fetch_known`,

`cached_latest`, and unknown latest-source states

• `weekly.local_schema` (newest local session used for fingerprinting)
• `weekly.schema_diff` and `evidence.schema_matches_baseline`
• `evidence.sample_freshness` and `evidence.fresh_evidence_source`
• `compatibility.latest_real_session_failure` when a prebump attempt failed
• `severity` and `recommendation` only as legacy escalation fields

3. **Usage / limits reading (Codex + Claude) — always verify every weekly run.** These drift independently of session schema (see §2), so a clean schema does **not** imply healthy usage reading. Each agent's `results.<agent>.weekly.probes` is a **list**; for every relevant entry confirm `ok == true` and `exit_code == 0`:

• Codex — `label == "codex_status_probe"` (parse `codex_status_json`): the

active CLI status channel (`five_hour`, `weekly` percent-left). The passive channel is the session JSONL `token_count` / `rate_limits` events, covered by the schema fingerprint above.

• Claude — `label == "claude_usage_probe"` (parse `claude_usage_json`): the

**authenticated** `/usage` reading (`session_5h`, `week_all_models`, `week_opus`). Also `label == "claude_status"` (parse `claude_status_json`): status.claude.com indicator/incidents. A failed or unparsed usage probe is a usage-format or auth regression even when versions match and the session schema is clean — never skip it, and report each probe's `ok` explicitly rather than collapsing it into the compatibility verdict.

Interpretation:

• `supports_latest`: latest known build is covered by

`evidence.fresh_evidence_source == "latest_prebump_report"` and `compatibility.latest_real_session_evidence == true` with `compatibility.latest_status == "current_fetch_known"`.

• `supports_installed_only`: installed build is covered by non-stale real local

evidence, but latest is newer, cached from a prior report, unknown, or lacks fresh real-session proof.

• `latest_unknown`: no configured/reachable latest source or no real-session

driver exists; do not claim latest support.

• `blocked_stale_sample`: evidence predates the installed CLI; run prebump before claiming support.
• `blocked_no_fresh_evidence`: a version changed but no fresh matching sample proves support.
• `format_drift_detected`: unknown schema/storage/usage fields appeared; update fixtures/parsers.
• `blocked_thin_sample`: the sample was both narrow and tiny, so it evidenced nothing either

way (§5a). Generate a session that actually uses tools — not a one-line prompt.

• `monitoring_broken`: latest source, usage probe, or discovery contract failed.
• `real_session_auth_failed` in blockers: the real-session driver ran but the

sandboxed agent was not authenticated; re-auth or provide the configured env token, then rerun prebump.

---

1a Real-Session Prebump Validation (required before latest claims)

Weekly scanning samples the newest on-disk session, which can predate a CLI upgrade and give a false "safe to bump" call (the codex 0.120.0 trap and the copilot `session.shutdown` trap). When weekly reports `recommendation == run_prebump_validator` — or before you stage any `max_verified_version` bump or latest-support claim — run the prebump path for every active agent being claimed. The driver exercises the currently installed CLI once inside a sandbox and diffs its output against the fixture baseline:

./scripts/agent_watch.py --mode prebump --agent codex --agent claude

Exit-code contract:

• `0` — every requested agent produced a fresh session and the schema

matches baseline. Safe to bump.

• `2` — at least one fresh session's schema does **not** match baseline.

Do **not** bump; investigate the schema diff in `scripts/probe_scan_output/agent_watch/<slug>-prebump/report.json`.

• `3` — at least one driver failed (timeout, auth, CLI not found, or

discovery contract violation — wrong session root, wrong glob, or missing required event types).

• `4` — config/invariant error: unknown `--agent` (or one with no

prebump block), missing/invalid `discover_session` contract, creden