/education-data-explorer

name: education-data-explorer
description: >-
  Discovers education data from Urban Institute Portal: endpoints, variables, year coverage, join keys (CCD, IPEDS, CRDC, Scorecard, SAIPE). Use to map questions to data. Load before education-data-query — discovery here, download there.
metadata:
  audience: research-planner
  domain: data-access

Education Data Explorer

Discovers available education data from the Urban Institute Education Data Portal: endpoints, variables, year coverage, and join keys for schools, districts, and colleges (CCD, IPEDS, CRDC, Scorecard, SAIPE, and more). Use during discovery and scoping phases when identifying what data exists, mapping research questions to endpoints, or resolving variable name discrepancies between documentation and actual field names. Load before education-data-query — this skill covers discovery; education-data-query handles the download.

Discover available education data from the Urban Institute Education Data Portal for research planning and query design.

What is the Education Data Portal?

• **Comprehensive education data** from Urban Institute - free and publicly available
• **Three data levels**: schools, school-districts, college-university
• **Multiple data sources**: CCD, IPEDS, CRDC, College Scorecard, EDFacts, SAIPE, FSA, MEPS, PSEO, etc.
• **Coverage**: 1980-2023 depending on source
• **Access**: Mirror downloads (parquet/CSV) via `education-data-query` skill
• **Documentation**: https://educationdata.urban.org/documentation/
• **Curation layer, not a direct pass-through**: The EDP standardizes data from original federal sources with lowercase variable names, integer-encoded categoricals, and uniform missing value codes (`-1`, `-2`, `-3`)
• **Coverage varies by source**: Some sources are fully mirrored; others are partially mirrored with only a subset of variables or datasets available. See individual `education-data-source-*` skills for source-specific coverage details

> **Skill Provenance Note:** Each `*-data-source-*` skill includes > `provenance.skill_last_updated` in its frontmatter. When exploring data > sources during Stage 2, note the provenance dates of any skills you > reference — if more than a few months old, flag this in your findings > so the orchestrator can consider re-verifying with data-ingest.

> **Note:** This workflow uses mirror-based file downloads, not paginated API calls. See `education-data-query` skill for fetch patterns and `datasets-reference.md` for file paths.

Reference File Structure

| File | Purpose | When to Read | |------|---------|--------------| | `schools-endpoints.md` | All school-level endpoints and variables | Researching K-12 schools | | `districts-endpoints.md` | All district-level endpoints and variables | Researching school districts | | `colleges-endpoints.md` | All college-level endpoints and variables | Researching higher education | | `variable-codes.md` | Code values for states, grades, race, etc. | Interpreting or filtering data | | `metadata-api.md` | Programmatic endpoint/variable discovery | Dynamic exploration |

Decision Trees

What data level do I need?

What entity am I researching?
├─ Individual K-12 schools → schools level
│   └─ See ./references/schools-endpoints.md
├─ School districts / LEAs → school-districts level
│   └─ See ./references/districts-endpoints.md
├─ Colleges / Universities → college-university level
│   └─ See ./references/colleges-endpoints.md
└─ Not sure
    ├─ Need school-specific data (discipline, AP, demographics) → schools
    ├─ Need aggregate district data (finance, poverty) → school-districts
    └─ Need postsecondary data (enrollment, aid, outcomes) → college-university

What topic am I researching?

Research topic?
├─ Enrollment / Demographics
│   ├─ K-12 public schools → CCD enrollment endpoints
│   ├─ Civil rights indicators → CRDC enrollment
│   └─ Colleges → IPEDS enrollment
├─ School Finance
│   ├─ District revenue/expenditure → CCD finance
│   └─ College finance → IPEDS finance
├─ Student Outcomes
│   ├─ K-12 assessments → EDFacts
│   ├─ Graduation rates (K-12) → EDFacts
│   ├─ College completion → IPEDS completions
│   └─ Post-college earnings → College Scorecard
├─ Student Aid / Loans
│   ├─ College financial aid → IPEDS aid
│   ├─ Federal loans/grants → FSA
│   └─ Debt/repayment → College Scorecard
├─ Discipline / Civil Rights
│   └─ K-12 discipline, harassment, restraint → CRDC
├─ Poverty Estimates
│   └─ District-level → SAIPE
└─ Directory / Location
    ├─ K-12 schools → CCD directory
    ├─ Districts → CCD directory
    └─ Colleges → IPEDS directory

How do I find specific variables?

Finding variables?
├─ Know the endpoint → Check reference file for variable list
├─ Know the topic → Use topic index below
├─ Need to search programmatically → See ./references/metadata-api.md
└─ Need code definitions → See ./references/variable-codes.md

Quick Reference: Data Levels

| Level | Key Sources | Primary ID | ID Format | |-------|-------------|------------|-----------| | schools | CCD, CRDC, EDFacts, MEPS, NHGIS | `ncessch` | 12-char string | | school-districts | CCD, SAIPE, EDFacts | `leaid` | 7-char string | | college-university | IPEDS, Scorecard, FSA, PSEO, EADA | `unitid` | 6-digit integer |

Quick Reference: Data Sources

| Source | Level | Description | Years | |--------|-------|-------------|-------| | CCD | Schools, Districts | Public K-12 directory, enrollment, finance | 1986-2023 | | CRDC | Schools | Civil rights indicators, discipline, AP courses | 2011-2021 | | EDFacts | Schools, Districts | Assessments, graduation rates | 2009-2020 | | IPEDS | Colleges | Enrollment, completions, finance, institutional data | 1980-2023 | | College Scorecard | Colleges | Earnings, debt, student outcomes | 1996-2020 | | SAIPE | Districts | Census poverty estimates for school-age children | 1995-2023 | | FSA | Colleges | Federal student aid, loans, grants, 90/10 | 1999-2021 | |