/education-data-context

name: education-data-context
description: >-
  Interpretation guidance for Urban Institute Portal datasets. Coded values (-1/-2/-3), year definitions, grade encoding, suppression, licensing, cross-source joins. Use when interpreting Portal data before analysis. Routes to source-specific skills.
metadata:
  audience: any-agent
  domain: data-documentation

Education Data Context

Data origin, caveats, and interpretation guidance for Urban Institute Education Data Portal datasets. Use when interpreting Portal coded values (-1/-2/-3 missing/not-applicable/suppressed), understanding year definitions (fall vs. academic year), applying correct grade encoding (grade=-1 means Pre-K, not missing), assessing suppression rates, citing data under ODC-By license, or reviewing any Portal data before analysis. Also covers joining identifiers across CCD, IPEDS, CRDC, and other sources, and routes to source-specific deep-dive skills.

This skill provides critical context for interpreting data from the Urban Institute Education Data Portal. Education data has source-specific limitations that can significantly affect analysis validity.

Why Data Context Matters

• **Source-specific limitations**: Each data source (CCD, IPEDS, CRDC, etc.) has unique constraints
• **Missing values have meaning**: Codes like -1, -2, -3 indicate specific conditions, not random missingness
• **Definitions change over time**: Variable definitions, categories, and coding schemes evolve
• **State comparisons require caution**: State-level data often cannot be directly compared
• **Citation is required**: The ODC Attribution License mandates proper citation
• **Skill provenance matters**: Each `*-data-source-*` skill includes `provenance.skill_last_updated` in its frontmatter. If this date is more than a few months old, treat the skill's claims about coded values, suppression patterns, and data quality with caution — data sources evolve and skill documentation may have drifted. Consider re-running data-ingest to re-verify.

Data Provenance: The Education Data Portal

All education data currently accessible through this system is obtained from the **Urban Institute Education Data Portal (EDP)**, not directly from original source agencies (NCES, Census Bureau, Department of Education, etc.). The EDP is a curation and standardization layer that:

• **Renames variables** to lowercase (e.g., `enrollment` not `MEMBER`)
• **Re-encodes categoricals** as integers (e.g., `1` not `"Regular school"`)
• **Standardizes missing values** using codes `-1` (missing), `-2` (not applicable), `-3` (suppressed)
• **May subset** each source's full variable catalog — not all variables from the original source are necessarily available through the Portal

Each `education-data-source-*` skill documents what is available through the Portal for that source, including any known gaps relative to the original data collection. When a skill also documents variables or components only available from the original source directly, this is clearly noted.

> **Note:** This provenance applies specifically to the current education data source skills. Future data source skills may access data from other providers with different characteristics.

Reference File Structure

Quick Context (This Skill)

| File | Content | When to Read | |------|---------|--------------| | `./references/ccd-context.md` | K-12 schools/districts caveats | After pulling CCD data | | `./references/ipeds-context.md` | College/university caveats | After pulling IPEDS data | | `./references/crdc-context.md` | Civil rights data caveats | After pulling CRDC data | | `./references/scorecard-context.md` | College Scorecard caveats | After pulling Scorecard data | | `./references/edfacts-context.md` | Assessment/graduation caveats | After pulling EDFacts data | | `./references/data-relationships.md` | Joining tables, identifiers | When merging datasets |

Deep-Dive Source Skills (Comprehensive Documentation)

These skills document both EDP-available data and original source context. Each skill notes when content applies only to the original source (not available through the Portal).

For comprehensive understanding beyond the quick context files above, load the dedicated data source skill:

| Data Source | Deep-Dive Skill | Key Deep Topics | |-------------|-----------------|-----------------| | CCD | `education-data-source-ccd` | Survey components, EDFacts submission, state variations, historical changes | | CRDC | `education-data-source-crdc` | Civil rights legal context, underreporting issues, year-to-year evolution | | EDFacts | `education-data-source-edfacts` | ESSA/NCLB context, why states aren't comparable, ACGR methodology | | IPEDS | `education-data-source-ipeds` | All 12+ surveys, graduation rate population limits, GASB vs FASB | | Scorecard | `education-data-source-scorecard` | IRS earnings methodology, Title IV selection bias, suppression rules | | SAIPE | `education-data-source-saipe` | Model-based estimation, no district confidence intervals | | FSA | `education-data-source-fsa` | Title IV programs, financial responsibility scores, 90/10 rule | | MEPS | `education-data-source-meps` | Superior to FRPL for cross-state poverty comparison | | NHGIS | `education-data-source-nhgis` | Census geography links, boundary changes over time | | NACUBO | `education-data-source-nacubo` | Endowment study methodology, voluntary participation bias | | NCCS | `education-data-source-nccs` | Form 990 data, NTEE codes, private college relevance | | EADA | `education-data-source-eada` | Title IX context, not same as compliance data | | Campus Safety | `education-data-source-campus-safety` | Clery Act, underreporting, geography definitions | | PSEO | `education-data-source-pseo` | LEHD methodology, experimental status, state coverage |

**When to load deep-dive skills:**

• Need to understand data collection methodology in detail
• Analyzing historical trends and need to know about definitio