/evidence-calibration-review

name: evidence-calibration-review
description: "Use when you want a per-claim evidence-tier audit on a text artifact before it ships — assign T1-T6 tiers to every load-bearing claim, surface calibration mismatches (high confidence on weak evidence, or honesty-theater under-claiming), and flag P11 (citation-as-decoration), P17 (pile-of-anecdotes-as-evidence), P54 (unverifiable single-source) patterns. Encodes the Evidence & Calibration deliberator role from the agent-council 5-perspective quality gate. Use standalone for fast evidence audit, or compose with the other 4 deliberator skills."
version: "0.1.1"
type: "codex"
tags: ["Quality Gate", "Evidence", "Calibration", "Council Deliberator"]
created: "2026-05-29"
valid_until: "2026-11-29"
derived_from: "prompts/evidence.md in Avyayalaya/agent-council"
tested_with: ["Claude Sonnet 4.6", "Claude Opus 4.6", "GPT-4o"]
license: "MIT"
composes_with:
  - package: "agent-council"
    skill: "skeptic-review"
    relation: "use_together"
    reason: "Skeptic surfaces unaddressed counter-positions; Evidence & Calibration tiers the claims those positions rest on. Together they catch weak-evidence claims masquerading as load-bearing."
  - package: "agent-council"
    skill: "adjudicator-synthesis"
    relation: "produces_input_for"
    reason: "Per-claim tier map + calibration issues feed directly into the Adjudicator's verdict policy."
  - package: "pm-skills"
    skill: "discovery-research"
    relation: "use_after"
    reason: "Discovery & Research produces evidence-graded findings; Evidence & Calibration audits whether downstream artifacts maintain the tiers honestly."
  - package: "pm-skills"
    skill: "competitive-market-analysis"
    relation: "use_after"
    reason: "Competitive analyses are claim-dense; Evidence & Calibration catches T5/T6 claims presented as T2 conclusions."
  - package: "pm-skills"
    skill: "metric-design-experimentation"
    relation: "use_after"
    reason: "Metric and experiment claims are particularly prone to over-claiming (sample → population). Evidence & Calibration catches these."
capability_summary: "Produces a structured Evidence & Calibration critique of a text artifact: per-claim tier assignment on the standard T1-T6 scale (with quoted claim + status + fix), calibration issues (high-confidence-on-weak-evidence and under-claimed-on-verified inversions), P11/P17/P54 pattern flags, would_block + irreducible flags. Output is fenced JSON suitable for downstream verdict aggregation."
input_schema:
  artifact: "string or path — the text artifact to audit (prose with claims clearly identifiable)"
  artifact_type: "string — optional, e.g., 'spec', 'memo', 'analysis', 'pitch'"
  domain_context: "object or path — optional, domain-specific evidence-tier rubric overrides"
  prior_round_critiques: "object — optional, all 4 R1 critiques from other deliberators when running Round 2 cross-read rebuttal"
output_schema:
  role: "Constant: evidence"
  round: "1 (independent critique) or 2 (cross-read rebuttal)"
  score: "1-5 where 1 = every load-bearing claim is unsourced or mis-tiered, 5 = every load-bearing claim is correctly tiered and well-calibrated"
  claim_tier_map: "Array of {claim, tier, status, fix} for each load-bearing claim"
  calibration_issues: "Array of strings naming over-claimed and under-claimed sentences"
  p11_p17_p54_flags: "Array of strings naming detected patterns (citation-as-decoration, pile-of-anecdotes, unverifiable single-source)"
  would_block: "Boolean — true if any load-bearing claim is T5/T6 without acknowledgment, or calibration inverted"
  irreducible: "Boolean — true only if evidence base is too thin for the claims being made; restructure required"
  notes: "≤2 sentences on overall evidence posture"
example_invocation: "examples/evidence-on-pricing-claim.md"

Purpose

Run a per-claim evidence-tier audit on a text artifact before it ships. The Evidence & Calibration role reads the artifact claim by claim and asks one question per claim: **what tier of evidence supports it, and is the artifact's stated confidence consistent with that tier?**

A claim asserted with high confidence on Tier 6 (inferred) evidence is a calibration failure. A claim hedged with "perhaps" when the evidence is Tier 1 (primary source, verified) is also a calibration failure — under-claiming is its own honesty failure. The skill catches both directions.

This is the boring and the load-bearing role on the panel. "Where is the source for X?" is the question that ends careers. Evidence & Calibration surfaces every unsourced claim before it ships.

The skill encodes the Evidence & Calibration role from the `agent-council` 5-deliberator quality gate. Use standalone for fast evidence audit, or compose with the other 4 deliberator skills for fuller coverage.

When to Use / When NOT to Use

**Use this skill when:**

• A claim-dense artifact (analysis, memo, public pitch) is about to ship and you want every load-bearing claim tiered
• You suspect over-claiming (high confidence on weak evidence) or under-claiming (hedging what is actually verified) and want both directions surfaced
• A piece relies on attributions ("X said Y" / "Microsoft did Z") and you want each verified or hedged appropriately
• You need to catch P11 (citation-as-decoration), P17 (pile-of-anecdotes-as-evidence), or P54 (unverifiable-single-source) patterns explicitly
• You are running a multi-deliberator review and need the Evidence & Calibration seat filled

**Do NOT use this skill when:**

• You need a structural critique (load-bearing claims, counter-positions) — use `skeptic-review` instead
• You need a voice critique (banned patterns, register check, CXO test) — use `voice-identity-review` instead
• You need a strategic-fit check (goal alignment, opportunity cost) — use `strategy-stakes-review` instead
• The artifact has no factual claims (a pure brainstorming note or speculative framing)
• You want the operator's own source-finding work done fo