/empirical-playbook

name: empirical-playbook
argument-hint: "<method, research question, or diagnostic>"
description: >-
  This skill covers applied microeconomic empirical methods and research design. Use when the user is selecting an identification strategy, comparing estimators, running diagnostics, designing a research study, or evaluating an empirical strategy. Triggers on "which method", "what estimator", "how to choose", "method comparison", "empirical strategy", "research design", "applied micro", "identification strategy", "power analysis", "design-based", "model-based", "minimum detectable effect", "specification".

Applied Micro Toolkit

Reference for applied micro research design: method selection, diagnostics, inference, pitfalls, reporting standards, and power analysis.

When to Use This Skill

Use when the user is:

• Choosing between empirical methods for a causal question
• Evaluating which identification strategy fits their data and setting
• Running standard diagnostic tests and unsure which ones apply
• Designing a study and needs to calculate statistical power
• Reviewing or critiquing an empirical strategy
• Preparing the "Empirical Strategy" section of a paper
• Downloading macroeconomic or cross-national data (see `references/data-sources.md` for FRED/World Bank API access)

Skip when:

• Implementation details for a specific method are needed (use `causal-inference` skill for IV, DiD, RDD, SC, matching)
• The task is structural estimation (use `structural-modeling` skill)
• The task is manuscript preparation or journal logistics (use `submission-guide` skill)
• The task is formal identification proof (use `identification-proofs` skill)
• The task is Bayesian model specification (use `bayesian-estimation` skill)

After selecting a method, the `econometric-reviewer` agent can review the implementation and the `identification-critic` agent can evaluate the identification argument.

Method Selection Decision Tree

Start with the fundamental question: **What source of variation identifies the causal effect?**

Step 1: What is your source of variation?

| Source of Variation | Method Family | Key Assumption | |--------------------|---------------|----------------| | Randomized assignment (with full compliance) | Experimental analysis (OLS on treatment indicator) | Random assignment | | Randomized assignment (with imperfect compliance) | IV / 2SLS using random assignment as instrument | Exclusion restriction, monotonicity | | Policy change at a sharp threshold | Sharp RDD | Continuity of potential outcomes at cutoff | | Policy change at a threshold with imperfect compliance | Fuzzy RDD (= IV at the cutoff) | Continuity + monotonicity at cutoff | | Policy change at a point in time, with affected and unaffected groups | Difference-in-differences | Parallel trends | | Staggered policy adoption across units over time | Staggered DiD (Callaway-Sant'Anna, Sun-Abraham, etc.) | Parallel trends (conditional on group and time) | | Rare event affecting a single unit, long pre-treatment data | Synthetic control | Pre-treatment fit implies post-treatment counterfactual | | Exogenous shifter of treatment that does not affect outcome directly | IV / 2SLS / GMM | Exclusion restriction, relevance, monotonicity | | Rich set of observables that plausibly captures all confounders | Matching, IPW, AIPW (selection on observables) | Conditional independence (no unobserved confounders) | | No credible exogenous variation | Sensitivity analysis, bounds, partial identification | Depends on bounding assumptions |

Step 2: Refinements Within Method Families

**Within DiD:**

Is treatment timing staggered?
├── No → Classic 2x2 DiD (TWFE is fine)
└── Yes
    ├── Can treatment turn off (reversals)?
    │   ├── Yes → de Chaisemartin-D'Haultfoeuille (2020)
    │   └── No
    │       ├── Do you have never-treated units?
    │       │   ├── Yes → Callaway-Sant'Anna (2021) with never-treated controls
    │       │   └── No → Callaway-Sant'Anna with not-yet-treated controls
    │       │           or Sun-Abraham (2021)
    │       └── Are effects likely heterogeneous across cohorts?
    │           ├── Yes → Callaway-Sant'Anna or Sun-Abraham (NOT TWFE)
    │           └── No → TWFE is OK, but report Bacon decomposition

**Within IV:**

How many instruments for how many endogenous regressors?
├── Exactly identified (K instruments = K endogenous)
│   └── 2SLS (= IV = Wald estimator for single instrument)
├── Over-identified (K instruments > K endogenous)
│   ├── 2SLS (default)
│   ├── GMM (efficient, use if heteroskedasticity suspected)
│   └── LIML (less biased with weak instruments)
└── Under-identified (K instruments < K endogenous)
    └── Cannot identify all parameters — need more instruments or fewer endogenous regressors

**Within RDD:**

Does crossing the threshold guarantee treatment?
├── Yes → Sharp RDD
└── No → Fuzzy RDD
    └── Is the running variable continuous?
        ├── Yes → Standard rdrobust
        └── No (discrete / few mass points)
            └── Cattaneo-Idrobo-Titiunik (2019) discrete RD methods

**Within Matching / Selection on Observables:**

Is the selection-on-observables assumption plausible?
├── No → Need a different identification strategy
└── Yes
    ├── Do you need ATE or ATT?
    │   ├── ATE → IPW or AIPW
    │   └── ATT → Matching or IPW with ATT weights
    ├── Is the propensity score model well-specified?
    │   ├── Uncertain → Use AIPW (doubly robust)
    │   └── Confident → IPW or regression adjustment
    └── Many covariates or nonlinear confounding?
        ├── Yes → ML-based methods (causal forests, DML)
        └── No → Parametric PS model + AIPW

Standard Diagnostics by Method

Key diagnostics to run for each method family. For full reporting checklists and minimum standards, see `references/reporting-standards.md`.

| Method | Must-Run Diagnostics | Key Concern | |--------|---------------------|-------------| | IV / 2SLS | Fi