/amann-evidence-standards

name: amann-evidence-standards
description: Use when appraising the cumulative evidence and ensuring balance in an Academy of Management Annals (Annals) review — weighing conflicting findings by credibility, steelmanning rival schools, and handling the author's own work even-handedly. Audits evidence quality and fairness; it does not design the framework (amann-organizing-framework) or build exhibits (amann-tables-figures).

Evidence Standards — Appraisal & Balance (amann-evidence-standards)

When to trigger

• The framework is set and you are filling cells with the literature
• The field has rival schools, a live controversy, or conflicting empirical results
• You must decide whether to cite *everything* or curate, and how to weigh disagreeing studies
• You are a contributor to this literature and worry the review tilts toward your own work

At Annals you are the field's reviewer-of-record

An Annals review carries no estimates of its own — but it is not therefore neutral. The "attitude" is **disciplined critical appraisal**: you judge how good the cumulative evidence is, reconcile conflicts by credibility, and state where the field's confidence is and is not warranted. This is the review-craft replacement for primary-research robustness checks: you are appraising *other people's* designs, not defending your own.

Completeness vs. selectivity: the Annals contract

A review must be **comprehensive in coverage** yet **selective in emphasis** — long (~50 pages) but not an inventory. Resolve the tension by **tiering** the corpus:

| Tier | Treatment | |------|-----------| | **Foundational / field-defining** | discussed in text — what it established and its limits | | **Important contributions** | grouped and weighed within the framework's cells; cited with their finding | | **Confirmatory / incremental** | cited in clusters ("see also …") to show coverage without bloating prose | | **Tangential** | cited only where it bears on a specific claim |

Comprehensiveness is proven by the *citation set* (the saturation log from `amann-literature-synthesis`); selectivity is exercised in the *prose*. Equal-length summaries of every paper abdicate the editorial judgment that is the review's value.

Weighing conflicting evidence (not vote-counting)

Management findings conflict constantly. Reconcile them by **why they differ**, never by tally:

• **Weigh by credibility.** Design strength (causal vs. correlational), sample, measurement validity, replication, and publication-bias risk — not "9 studies positive, 4 negative."
• **Separate by construct and context.** Studies often disagree because they measure different constructs or run in different settings; surface that rather than forcing a false consensus.
• **Flag the cumulative-evidence quality.** State plainly where the evidence is strong, mixed, or thin — and where apparent consensus rests on a few non-independent studies.
• **Name what would settle it.** For a live debate, specify the study design that would resolve it (this seeds the agenda).

When appraisal needs an actual number (execution bridge)

Occasionally a credibility judgment cannot be settled by reading alone: the review's account of a controversy hinges on whether a staggered-adoption TWFE estimate survives modern corrections, or an apparent consensus may melt once publication bias is priced in. When a load-bearing magnitude comes with a replication package, audit it rather than adjudicate by prose — the shared playbook [`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md) maps each design family to callable StatsPAI / Stata MCP tools (`bacon_decomposition` to expose bad-comparison weighting, `callaway_santanna` to re-estimate, `honest_did_from_result` for pre-trend fragility). Any number produced this way must come from an actual run and be labelled as the review's own re-analysis. This is the exception, not the Annals default: most appraisal here stays qualitative.

Fairness across schools and the self-citation trap

Annals is the **review-of-the-field**: its account of a debate becomes the field's shared reference, and **the surveyed authors often referee the review**. Balance is therefore both ethical and strategic.

• **Steelman every camp.** State each school's strongest case in terms its proponents would accept *before* critiquing it. Refute positions at their best.
• **Attribute ideas to originators**, not popularizers — a recurring referee complaint.
• **Audit self-citation.** Your own work is cited at the tier its *importance to the field* warrants — no more. Rivals get their strongest statement. A reader who does not know who wrote the review cannot tell from the emphasis.
• **Label your read as your read.** Where you take a provocative position (the "attitude"), mark it as the author's judgment, not as settled consensus.

Checklist

• [ ] Corpus tiered (foundational / important / confirmatory / tangential); prose emphasis matches tier
• [ ] Comprehensiveness provable from the citation set + saturation log
• [ ] Conflicting findings reconciled by credibility, construct, and context — not vote-counting
• [ ] Cumulative-evidence quality stated (strong / mixed / thin) per claim
• [ ] Every rival school stated at its strongest before critique (steelman)
• [ ] Idea attribution traces to originators
• [ ] Self-citation audited: own work at field-warranted tier; emphasis identity-blind
• [ ] Provocative positions labelled as the author's read; what-would-settle-it named for live debates
• [ ] No important author/school an informed referee could say was slighted or omitted

Anti-patterns

• Comprehensiveness theatre: equal-length summaries of every paper (no editorial judgment)
• Vote-counting conflicting results instead of weighing credibility, construct, and context
• Treating all findings as equally credible regardless of design quality
• Strawmanning the camp the author disagrees with (the referee may be that