/schema-design

name: schema-design
description: Decide what shape the knowledge takes in this project — facts, rules, stories, wiki entries, screenplays, custom. Use whenever the user is starting a project, the app-type definition section of PLAN.md is unsettled, you're about to enter the extract phase without a clear target schema, or anyone says "what format should we use?". Schema decisions cascade — get this loose enough to iterate but specific enough to write a starter extractor.
metadata:
  triggers:
    - design the schema
    - schema design
    - what format
    - knowledge schema
    - app-type definition
    - knowledge format
    - structure the knowledge

schema-design

This is the most consequential decision in the knowledge phases. Get it wrong-or-too-rigid and every downstream phase pays for it (an earlier production system is a cautionary tale — see `references/sku-regression-case-study.md`). Get it right and the rest of the pipeline becomes obvious.

What schema-design is NOT

• It's not picking from a closed menu. There is no "the John schema." Different projects want different shapes.
• It's not a one-shot decision. The schema **evolves** through the early phases as the corpus reveals itself.
• It's not a JSON spec for the John-equipped agent to autocomplete. It's a *taste call* the user owns.

The app-type definition cascade

The app-type definition cascade — knowledge format → knowledge schema → app mechanism → build pipeline — has each link constrain the next. Schema is the second link in the chain — downstream of *what kind of knowledge* and upstream of *what the app does*. The cascade itself is explained in `references/app-type-definition-cascade.md` and applied in [[plan-md-authoring]]; this skill is where the *schema link* gets designed.

You make schema decisions **only after the format decision is roughly settled**. Reverse the order and you end up over-fitting the schema to the corpus, then re-doing it when the runtime asks for something the schema can't represent.

Before you design — read the corpus first

John's job is to *teach* schema design, not hand you a schema: abstract the methodology from prior projects — what to consider, when and how to iterate — and apply it here. That methodology starts with **reading what's actually in the corpus** before sketching a schema. Pre-designing in a vacuum is how you get over-fit or under-specified schemas.

Practical method:

1. **Read [[parsing]]'s output**. Walk through a representative sample of `<project>/.john/parsed/*/doc.md` (don't read everything; read enough to recognize patterns). 2. **Ask survey questions** as you read:

• Is the corpus mostly *atomic statements* (factual)?
• Mostly *prescriptive how-to* (procedural / rules)?
• Mostly *narrative* (storylines / characters / scenes)?
• Heavy on *connections between entities* (relational / wiki)?
• Mixed? Which mix?

3. **Notice structural features** the corpus already exhibits: causal chains, taxonomies, glossary-shaped terminology, recurring entities, citations, decision flowcharts. 4. **Cross-reference user intent** from PLAN.md's project intent + app mechanism (the app-type definition cascade). A corpus full of facts might suit a quiz app (procedural runtime) OR a wiki (browsable runtime); user intent decides. 5. **THEN sketch the schema** to fit (corpus shape × user intent), not to fit a default.

Skip this survey and you'll re-do the schema mid-extraction. Cheap to do early; expensive to fix late.

Format options (the menu, but it's open)

Common forms knowledge takes:

• **Facts** (atomic statements with citations): for encyclopedic projects, briefings, knowledge bases
• **Rules** (trigger + decision + action): for verification, compliance, business-logic apps
• **Skills** (Claude Code-style how-to procedures): for distillation projects, where final output is itself reusable
• **Wiki entries** (long-form, cross-linked, browsable): for navigable knowledge products
• **Storylines** (character + event + setting + branching): for narrative/game projects
• **Screenplays** (scene + dialogue + direction): for content production
• **Graphs** (entities + typed relationships): for structured-data products
• **Custom**: for projects that don't fit. The user defines.

A project can have **multiple formats** — e.g., facts + skills + glossary, or storylines + character profiles. They don't have to share a schema.

Schema shape per format

Design the schema for *this* corpus — there's no canonical shape to fill in. For illustrations of the *kind* of fields each common format tends to want (facts, rules, skills, wiki, storylines, screenplays, graphs), see `references/starter-schemas.md`. Read them as examples, not a menu. Header + body progressive disclosure stays universal regardless of which format you land on.

The MECE principle

Extract "everything there is" OR "everything needed for what" — Mutually Exclusive, Collectively Exhaustive within the chosen schema. **Don't extract the same fact three times under different schemas; don't leave the user's input partially covered.**

MECE applies to coverage, not granularity — a fact and a rule that depends on the fact can coexist if they're in different formats. Inside a single format, no duplicates.

Header + body — a universal layer above schema

Every entry, regardless of format, gets a two-tier structure:

• **Header**: one-line description + classification + cross-references. Pinned in any retrieval context; cheap to load.
• **Body**: full content. Loaded on demand when an extractor or runtime consumes the entry.

This is **not a schema choice** — it's a universal practice applied on top of whichever schema you design. The schema defines what fields the *body* has; the header is always present, always one-line + classification + refs. Enforced during [[knowledge-rewrite]]. Design the schema with this split assumed.

The schema pilot — test before mass production

A schema that is