/quality-assessment

name: quality-assessment
description: Evaluate ENCODE experiment quality using standard metrics and audit flags. Use when the user asks about data quality, wants to filter for high-quality experiments, needs to interpret quality metrics (FRiP, NSC, RSC, NRF, IDR, TSS enrichment, fragment size), wants to understand ENCODE audit warnings, needs to compare quality across experiments, or is deciding whether data is usable for their analysis. Also use when the user mentions QC, quality control, or data filtering.

Assess ENCODE Data Quality

When to Use

• User asks about data quality, QC metrics, or whether an experiment is reliable
• User wants to filter experiments by quality (FRiP, NSC, RSC, NRF, IDR, TSS enrichment)
• User asks "is this experiment good enough?" or "should I use this data?"
• User needs to interpret ENCODE audit flags (ERROR, NOT_COMPLIANT, WARNING)
• User wants to compare quality across multiple experiments
• User is selecting high-quality experiments for a meta-analysis or aggregation

Help the user evaluate whether ENCODE experiments meet quality standards for their analysis. Quality assessment is not a single-metric exercise — it requires integrating multiple orthogonal measures in the context of the specific assay, biological system, and analytical goals.

Literature Foundation

| # | Reference | Key Contribution | |---|-----------|-----------------| | 1 | Landt et al. 2012, Genome Res, DOI:10.1101/gr.136184.111 (~3,500 cit) | ENCODE/modENCODE ChIP-seq guidelines; defined NSC, RSC, NRF, FRiP thresholds | | 2 | ENCODE Project Consortium 2020, Nature, DOI:10.1038/s41586-020-2493-4 (~1,656 cit) | ENCODE Phase 3; expanded quality standards to new assays, defined cCRE registry | | 3 | Buenrostro et al. 2013, Nat Methods, DOI:10.1038/nmeth.2688 (~7,000 cit) | Introduced ATAC-seq; established fragment size and TSS enrichment as key QC | | 4 | Ou et al. 2018, BMC Genomics, DOI:10.1186/s12864-018-4559-3 | ATACseqQC R package; systematic quality metrics for ATAC-seq | | 5 | Conesa et al. 2016, Genome Biol, DOI:10.1186/s13059-016-0881-8 (~2,363 cit) | RNA-seq best practices survey; defined mapping rate, rRNA, gene body coverage | | 6 | Foox et al. 2021, Genome Biol, DOI:10.1186/s13059-021-02529-2 | SEQC2 EpiQC consortium; multi-platform WGBS benchmarking | | 7 | Yardimci et al. 2019, Genome Biol, DOI:10.1186/s13059-019-1658-7 | Hi-C quality measures; cis/trans ratio, distance-dependent decay, resolution | | 8 | Skene & Henikoff 2017, eLife, DOI:10.7554/eLife.21856 (~1,800 cit) | CUT&RUN method; established spike-in normalization and low-background QC | | 9 | Kaya-Okur et al. 2019, Nat Commun, DOI:10.1038/s41467-019-09982-5 (~1,200 cit) | CUT&Tag method; tagmentation-based profiling with distinct QC profile | | 10 | Li et al. 2011, Ann Appl Stat, DOI:10.1214/11-AOAS466 (~1,500 cit) | Irreproducible Discovery Rate (IDR); principled replicate concordance | | 11 | Hitz et al. 2023, Nucleic Acids Res, DOI:10.1093/nar/gkad243 | ENCODE uniform processing pipelines; standardized QC across all assays | | 12 | Nordin et al. 2023, Genome Biol, DOI:10.1186/s13059-023-03027-3 | CUT&RUN suspect list; identified artifact-prone regions specific to CUT&RUN/CUT&Tag | | 13 | Amemiya et al. 2019, Sci Rep, DOI:10.1038/s41598-019-45839-z (~1,372 cit) | ENCODE Blacklist v2; artifact regions to exclude from all analyses |

Step 1: Retrieve Experiment Details and Audit Status

Use `encode_get_experiment` with the accession to get full metadata including:

• **Audit status** (ERROR, NOT_COMPLIANT, WARNING, INTERNAL_ACTION)
• **Replicate information** (biological and technical replicates)
• **Pipeline and analysis details** (which ENCODE uniform pipeline was used)
• **Quality metrics** embedded in file objects

encode_get_experiment(accession="ENCSR...")

For batch assessment across multiple experiments:

encode_search_experiments(assay_title="...", organ="...", limit=50)
# Then iterate through results checking audit flags

Step 2: Interpret ENCODE Audit Flags

ENCODE audits are generated by automated validators during the ENCODE uniform processing pipeline (Hitz et al. 2023). They flag experiments by severity:

| Level | Meaning | Action | |-------|---------|--------| | **ERROR** | Critical issues — data may be unreliable | Avoid using unless no alternative exists. Document thoroughly if used. | | **NOT_COMPLIANT** | Does not meet current ENCODE standards | Usable with caveats. Check which specific standard is violated. | | **WARNING** | Minor issues detected | Generally safe. Document the specific warning. | | **INTERNAL_ACTION** | DCC processing notes | Usually not a concern for external users. |

**Common audit categories and what they mean:**

| Audit Category | What It Checks | |---------------|---------------| | `replicate concordance` | IDR or correlation between biological replicates | | `library complexity` | NRF, PBC1, PBC2 — whether library is saturated | | `read depth` | Whether minimum depth thresholds are met | | `control quality` | Whether input/IgG control is adequate | | `mapping quality` | Alignment rate and uniquely mapped fraction | | `peak calling` | Whether peaks were called successfully, FRiP | | `antibody validation` | Whether antibody meets ENCODE standards |

Present every audit flag to the user and explain each one. A single ERROR audit does not automatically disqualify an experiment — context matters.

Step 3: Evaluate ChIP-seq Quality (Landt et al. 2012)

The ENCODE ChIP-seq guidelines (Landt et al. 2012) established the foundational metrics still used today. These were developed from analysis of hundreds of ChIP-seq experiments and reflect empirically-derived thresholds.

Core Metrics

| Metric | Threshold | Concern | What It Measures | Why It Matters | |--------|-----------|---------|-----------------|----------------| | **FRiP** | ≥1% (TF), ≥5% (histone) | Below threshold | Fraction of reads in peaks | Signal enrichment. Very low