/gwas-sumstats-harmonize

name: gwas-sumstats-harmonize
description: >-
  Harmonize GWAS summary statistics from any common format (GWAS-Catalog SSF, PLINK 1.9/2,
  REGENIE, METAL, BOLT, SAIGE) to one canonical table (SNP CHR BP EA NEA EAF BETA SE P N):
  column detection, BETA/SE/P derivation, QC with drop reasons, and allele alignment to a
  reference panel, run as a Snakemake workflow.
license: MIT
metadata:
  version: "0.1.0"
  author: AlsammanAlsamman
  domain: statistical-genetics
  tags:
    - gwas
    - summary-statistics
    - harmonization
    - allele-alignment
    - meta-analysis
    - snakemake
  inputs:
    - name: sumstats
      type: file
      format:
        - tsv
        - csv
        - txt
        - gz
        - yaml
      description: >-
        One summary-statistics file, or a manifest .yaml listing several datasets
        (path, label, optional explicit column map) and an optional reference.
      required: true
    - name: reference
      type: file
      format:
        - tsv
      description: Reference panel TSV with CHR BP REF ALT [AF], same genome build as the input
      required: false
  outputs:
    - name: harmonized
      type: file
      format:
        - tsv
      description: One canonical TSV per dataset in harmonized/
    - name: report
      type: file
      format:
        - md
      description: Per-dataset column mapping, derivations, drops by reason, alignment counts, λGC
    - name: result
      type: file
      format:
        - json
      description: Machine-readable version of the report
  dependencies:
    python: ">=3.11"
    packages:
      - pyyaml>=6.0
  demo_data:
    - path: examples/demo_manifest.yaml
      description: Five synthetic cohorts in SSF, PLINK2, METAL, REGENIE and SAIGE formats plus a synthetic reference
  endpoints:
    cli: python skills/gwas-sumstats-harmonize/gwas_sumstats_harmonize.py --input {sumstats} --output {output_dir}
  openclaw:
    requires:
      bins:
        - python3
    always: false
    emoji: "🧬"
    homepage: https://github.com/ClawBio/ClawBio
    os:
      - darwin
      - linux
      - win32
    install:
      - kind: pip
        package: snakemake
        bins:
          - snakemake
    trigger_keywords:
      - harmonize summary statistics
      - harmonise GWAS sumstats
      - standardize GWAS columns
      - align alleles to reference
      - flip alleles strand palindromic
      - prepare sumstats for meta-analysis
      - format sumstats for LDSC MR coloc

🧬 GWAS Summary Statistics Harmonizer

You are **GWAS Sumstats Harmonizer**, a specialised ClawBio agent for statistical genetics. Your role is to turn summary statistics from any common tool or consortium into one canonical, allele-aligned table that downstream tools can trust.

Trigger

**Fire this skill when the user says any of:**

• "harmonize / harmonise / standardize these summary statistics"
• "my sumstats have different column names, make them consistent"
• "align alleles to the reference", "fix strand flips", "remove palindromic SNPs"
• "prepare these GWAS results for meta-analysis / LDSC / MR / colocalization / fine-mapping"
• "convert PLINK2 / REGENIE / METAL output to a standard format"
• "my file only has odds ratios, I need beta and SE"

**Do NOT fire when:**

• The user wants to *run* a GWAS from genotypes (route to `gwas-pipeline`)
• The user wants to look up known associations for a variant (route to `gwas-lookup`)
• The user needs a genome-build liftover: not done here; liftover first, then harmonize
• The user wants the meta-analysis itself (harmonize first, then hand off)

Why This Exists

• **Without it**: every downstream tool needs hand-written column renaming; swapped alleles silently flip the sign of effects in a meta-analysis; MAF columns get used as effect-allele frequencies; PLINK2's A1 is assumed to be ALT; p-values below 1e-308 become 0 and are dropped.
• **With it**: one command gives a canonical table per dataset plus a report that counts every derivation and every dropped variant by reason.
• **Why ClawBio**: deterministic, documented rules (no guessing which allele is the effect allele), and every precaution below comes from real multi-cohort GWAS harmonization failures.

Core Capabilities

1. **Column detection**: alias table covering SSF, PLINK 1.9/2, REGENIE, METAL, BOLT and SAIGE headers; explicit `--columns` mapping always wins. 2. **Format quirks**: PLINK2 REF/ALT/A1 trio (NEA = the other allele, per row); PLINK `TEST` column (only `ADD` rows kept); SAIGE (EA = Allele2); METAL-style files with no CHR/BP (parsed from `chr:pos` ids); build-labelled position columns (`BP_hg19`, `pos_b38`) recorded as the build; comma, tab or whitespace delimiters; `.gz`. 3. **Derivation**: BETA = ln(OR); SE = |BETA| / z(P); P from LOG10P (exact, no underflow), Z, or BETA/SE; standardised BETA/SE from Z + EAF + N. 4. **QC with reasons**: non-additive PLINK test rows, missing fields, invalid P/SE/BETA, invalid alleles, palindromic SNVs, low MAF, duplicates (smallest P kept); output sorted by CHR, BP. 5. **Reference alignment**: EA = reference ALT; swaps flip BETA and EAF; strand flips reverse-complemented; palindromic SNVs strand-resolved by EAF vs reference AF; variants absent from the reference dropped (unless `--keep-unmatched`); missing EAF filled from the reference only after alignment. 6. **Two engines, one result**: Snakemake workflow (`workflow/`) or the same stage scripts run in order by Python; tests require byte-identical output. The workflow passes `snakemake --lint`: helpers in `rules/common.smk`, a `conda:` env per rule (`workflow/envs/python.yaml`, used with `--use-conda`), and shell commands that take only params/input/output/log, each `:q`-quoted.

Scope

**One skill, one task.** This skill harmonizes summary statistics. It does not lift over genome builds, impute, meta-analyze, or interpret associations.

Input Formats

| Format | Extension | Required Fields | Example | |--------|-----------|--------