/scientific-critical-thinking

name: scientific-critical-thinking
description: Evaluate scientific claims and evidence quality. Use for assessing experimental design validity, identifying biases and confounders, applying evidence grading frameworks (GRADE, Cochrane Risk of Bias), or teaching critical analysis. Best for understanding evidence quality, identifying flaws. For formal peer review writing use peer-review.
allowed-tools: Read Write Edit
license: MIT license
compatibility: Analytical guidance needs no network. Optional figures via the scientific-schematics skill require OPENROUTER_API_KEY and outbound API access to OpenRouter.
metadata:
  version: "1.2"
  skill-author: K-Dense Inc.

Scientific Critical Thinking

Overview

Critical thinking is a systematic process for evaluating scientific rigor. Assess methodology, experimental design, statistical validity, biases, confounding, and evidence quality using GRADE and Cochrane ROB frameworks. Apply this skill for critical analysis of scientific claims.

When to Use This Skill

This skill should be used when:

• Evaluating research methodology and experimental design
• Assessing statistical validity and evidence quality
• Identifying biases and confounding in studies
• Reviewing scientific claims and conclusions
• Conducting systematic reviews or meta-analyses
• Applying GRADE or Cochrane risk of bias assessments
• Providing critical analysis of research papers

Visual Aids (Optional)

Only add figures when the **user explicitly requests** a diagram (for example, a GRADE flowchart, bias decision tree, or evidence-quality framework).

**When figures help:**

• Critical thinking framework diagrams
• Bias identification decision trees
• Evidence quality assessment flowcharts
• GRADE or risk-of-bias evaluation frameworks

**How to create figures:**

• **Preferred:** Use the **scientific-schematics** skill for AI-generated diagrams from a natural-language description
• **Alternative:** Build figures in your usual tools (draw.io, PowerPoint, matplotlib, etc.)

From the `scientific-schematics` skill directory, with `OPENROUTER_API_KEY` set:

python scripts/generate_schematic.py "GRADE evidence assessment flowchart with downgrade and upgrade factors" -o figures/grade_flowchart.png --doc-type report

**Disclosure:** AI schematic generation sends your prompt to [OpenRouter](https://openrouter.ai/) (a third-party API). Do not include unpublished sensitive details unless that transmission is appropriate for your project.

---

Core Capabilities

Seven capability areas, each with the questions to ask and what the answers imply, are in [references/core_capabilities.md](references/core_capabilities.md):

1. **Methodology critique** — design, controls, confounding, and whether the method can answer the question asked. 2. **Bias detection** — selection, measurement, publication, and cognitive biases. 3. **Statistical analysis evaluation** — power, multiplicity, p-value misuse, effect sizes. 4. **Evidence quality assessment** — study hierarchy, replication, and strength of inference. 5. **Logical fallacy identification** — the fallacies that recur in scientific argument. 6. **Research design guidance** — how to strengthen a design before data collection. 7. **Claim evaluation** — separating what was shown from what is being asserted.

Per-topic detail is in [references/scientific_method.md](references/scientific_method.md), [references/common_biases.md](references/common_biases.md), [references/statistical_pitfalls.md](references/statistical_pitfalls.md), [references/evidence_hierarchy.md](references/evidence_hierarchy.md), [references/logical_fallacies.md](references/logical_fallacies.md), and [references/experimental_design.md](references/experimental_design.md).

Application Guidelines

General Approach

1. **Be Constructive**

• Identify strengths as well as weaknesses
• Suggest improvements rather than just criticizing
• Distinguish between fatal flaws and minor limitations
• Recognize that all research has limitations

2. **Be Specific**

• Point to specific instances (e.g., "Table 2 shows..." or "In the Methods section...")
• Quote problematic statements
• Provide concrete examples of issues
• Reference specific principles or standards violated

3. **Be Proportionate**

• Match criticism severity to issue importance
• Distinguish between major threats to validity and minor concerns
• Consider whether issues affect primary conclusions
• Acknowledge uncertainty in your own assessments

4. **Apply Consistent Standards**

• Use same criteria across all studies
• Don't apply stricter standards to findings you dislike
• Acknowledge your own potential biases
• Base judgments on methodology, not results

5. **Consider Context**

• Acknowledge practical and ethical constraints
• Consider field-specific norms for effect sizes and methods
• Recognize exploratory vs. confirmatory contexts
• Account for resource limitations in evaluating studies

When Providing Critique

**Structure feedback as:**

1. **Summary:** Brief overview of what was evaluated 2. **Strengths:** What was done well (important for credibility and learning) 3. **Concerns:** Issues organized by severity

• Critical issues (threaten validity of main conclusions)
• Important issues (affect interpretation but not fatally)
• Minor issues (worth noting but don't change conclusions)

4. **Specific Recommendations:** Actionable suggestions for improvement 5. **Overall Assessment:** Balanced conclusion about evidence quality and what can be concluded

**Use precise terminology:**

• Name specific biases, fallacies, and methodological issues
• Reference established standards and guidelines
• Cite principles from scientific methodology
• Use technical terms accurately

When Uncertain

• **Acknowledge uncertainty:** "This could be X or Y; additional information needed is Z"
• **Ask clarifying questions:** "Was [methodolog