/peer-review-methodology

name: peer-review-methodology
description: "Structured peer review of manuscripts and grants. 7-stage evaluation: initial assessment, section review, statistical rigor, reproducibility, figure integrity, ethics, writing. Covers CONSORT/STROBE/PRISMA and report structure. For evidence quality see scientific-critical-thinking; scoring see scholar-evaluation."
license: CC-BY-4.0

Scientific Peer Review

Overview

Peer review is a systematic process for evaluating scientific manuscripts and grant proposals. This knowhow covers the complete review cycle: from initial assessment through detailed section-by-section evaluation, statistical rigor checks, reproducibility assessment, figure integrity verification, ethical considerations, and writing quality — culminating in a structured review report with actionable feedback.

Key Concepts

1. Review Types and Expectations

| Review Type | Scope | Typical Length | Key Focus | |-------------|-------|---------------|-----------| | **Original research** | Full evaluation | 1–3 pages | Rigor, novelty, reproducibility | | **Review/Meta-analysis** | Coverage and bias | 1–2 pages | Completeness, search strategy, systematic approach | | **Methods paper** | Validation | 1–2 pages | Comparison to existing methods, reproducibility | | **Short communication** | Proportional | 0.5–1 page | Core findings rigor despite brevity | | **Grant proposal** | Feasibility + significance | 1–3 pages | Innovation, approach, team, budget justification |

2. Comment Severity Levels

• **Major comments**: Issues that significantly impact validity, interpretability, or significance. Must be addressed for publication. Examples: fundamental design flaws, unsupported conclusions, missing controls.
• **Minor comments**: Issues that improve clarity or completeness but don't affect core validity. Examples: unclear labels, missing methods details, grammatical errors.
• **Questions for authors**: Requests for clarification where the reviewer cannot evaluate without additional information.

3. Reporting Standards Quick Reference

| Standard | Applies to | Key Check | |----------|-----------|-----------| | **CONSORT** | Randomized controlled trials | Flow diagram, randomization, blinding, ITT analysis | | **STROBE** | Observational studies | Study design, setting, participants, variables, bias | | **PRISMA** | Systematic reviews / meta-analyses | Search strategy, PICO, risk of bias, forest plots | | **ARRIVE** | Animal research | Species, sample size, randomization, 3Rs | | **MIAME** | Microarray experiments | Platform, normalization, data deposit (GEO/ArrayExpress) | | **MINSEQE** | Sequencing experiments | Read counts, mapping, QC metrics, data deposit (SRA) |

Decision Framework

What are you reviewing?
├── Scientific manuscript
│   ├── Original research → Full 7-stage workflow
│   ├── Review / meta-analysis → Emphasize Stage 2 (Introduction + Methods) + Stage 4
│   ├── Methods paper → Emphasize Stage 3 (rigor) + Stage 4 (reproducibility)
│   └── Short communication → Abbreviated workflow (Stages 1, 2, 3, 7)
├── Grant proposal
│   └── Focus on: significance, innovation, approach feasibility, team qualifications
└── Not a document review
    └── Use scientific-critical-thinking for evidence evaluation

| Reviewer Situation | Focus Areas | Time Budget | |-------------------|-------------|-------------| | First-round journal review | All 7 stages, full detail | 4–8 hours | | Revision re-review | Only check if previous concerns addressed | 1–2 hours | | Internal lab feedback | Stages 1–3, 7 (skip ethics, reporting standards) | 2–4 hours | | Conference abstract | Stage 1 only + brief methods check | 30 min | | Grant review | Significance + innovation + approach + team | 3–6 hours |

Best Practices

1. **Read the whole manuscript once before writing any comments**: Resist the urge to annotate during first reading. Your initial impression informs the summary statement and helps distinguish major from minor issues.

2. **Separate description from judgment**: For each concern, first state what you observed ("The authors report p=0.04 without correction for 12 comparisons"), then state why it's problematic ("This inflates the false positive rate"), then suggest a fix ("Apply Bonferroni or FDR correction and re-evaluate significance").

3. **Number every comment for easy reference**: Both major and minor comments should be sequentially numbered so authors can address each one explicitly in their response letter.

4. **State your confidence level on statistical concerns**: If you are uncertain about a statistical issue, say so explicitly ("I am not certain whether the normality assumption holds here — the authors should verify with a Shapiro-Wilk test or use a non-parametric alternative").

5. **Check reporting standards compliance**: Before writing the review, identify which reporting standard applies (CONSORT, STROBE, PRISMA, etc.) and use its checklist to verify completeness. Missing checklist items are legitimate major comments.

6. **Acknowledge strengths explicitly**: Every review should include 2–3 specific strengths. This is not politeness — it tells editors which aspects are sound and helps authors understand what NOT to change during revision.

7. **Never request experiments beyond the study's scope**: A reviewer should point out limitations, not redesign the study. "This limitation should be acknowledged in the Discussion" is appropriate. "The authors should run a new cohort with 500 patients" is not.

8. **Be concrete about figure quality**: Instead of "figures need improvement," say "Figure 3A: y-axis label missing units; error bars are not defined in the legend (SD vs SEM?); color coding is not colorblind-accessible."

9. **Maintain anonymity and professionalism**: In single/double-blind review, avoid self-referential comments ("In our previous work..."). Never use dismissive or condescending language. Frame all criticism constructively.

10