/compaction

name: hipocampus-compaction
description: "Build 5-level compaction tree (daily/weekly/monthly/root) with smart thresholds and fixed/tentative lifecycle. Run at session start when triggers are met, or via external scheduler."

Memory Compaction Tree

5-level hierarchical index over raw memory logs. Compaction nodes are **search indices** — originals are never deleted.

Hierarchy

memory/
  ROOT.md                      <- root node (topic index, ~3K tokens, Layer 1)
  2026-03-15.md                <- raw daily log (permanent, append-only)
  daily/2026-03-15.md          <- daily compaction node
  weekly/2026-W11.md           <- weekly compaction node
  monthly/2026-03.md           <- monthly compaction node

**Compaction chain:** Raw → Daily → Weekly → Monthly → Root

**Tree traversal (search):** Root → Monthly → Weekly → Daily → Raw

Fixed vs Tentative Nodes

Every compaction node has a status:

• **tentative** — period is still ongoing, regenerated when new data arrives
• **fixed** — period ended, never updated again

# Indicated in YAML frontmatter
---
type: weekly
status: tentative
period: 2026-W11
---

**Key: tentative nodes are created immediately — ROOT.md is usable from day one.**

When to Run

Called from hipocampus:core Session Start step 7, or directly by an external scheduler (e.g., OpenClaw heartbeat). Check trigger conditions below.

Trigger Conditions

| Level | Tentative Create/Update | Fixed Transition | |-------|------------------------|-----------------| | Raw → Daily | On each new raw addition | Date changes | | Daily → Weekly | On daily add/change | ISO week ended + 7 days elapsed | | Weekly → Monthly | On weekly add/change | Month ended + 7 days elapsed | | Monthly → Root | On monthly add/change | Never (root accumulates forever) |

Smart Thresholds

Below threshold: copy/concat verbatim (no information loss). Above threshold: generate LLM keyword-dense summary.

| Level | Threshold | Above | Below | |-------|-----------|-------|-------| | Raw → Daily | ~200 lines | LLM keyword-dense summary | Copy raw verbatim | | Daily → Weekly | ~300 lines combined | LLM keyword-dense summary | Concat dailies | | Weekly → Monthly | ~500 lines combined | LLM keyword-dense summary | Concat weeklies | | Monthly → Root | Always | Recursive recompaction | (N/A) |

Type-Aware Compaction

Memory Types

Entries in daily logs are tagged: `## Topic [type]`. Four types exist:

| Type | Compaction behavior | |------|-------------------| | `user` | Always preserve core content. Never compress to Historical Summary. | | `feedback` | Always preserve rule + why + how-to-apply structure. Never compress to Historical Summary. | | `project` | Completed → compress to Historical Summary. Active → keep in Active Context. | | `reference` | Preserve pointer + 1-line description. Mark `[?]` if >30 days unverified. |

**Backward compat:** Untagged entries (no `[type]` in heading) → treat as `[project]`.

Topics Keyword Extraction

At every compaction level, extract topic keywords from content and write to frontmatter `topics` field:

• Scan headings (`## Topic [type]`) for keywords
• Scan Key Decisions for decision keywords
• Include type tag: `topics: [hipocampus [project], terse-responses [feedback]]`

Exclusion Filtering

When generating LLM summaries, strip:

• Code blocks (triple backtick) → replace with `→ filepath:lines`
• Stack traces → 1-line error message
• Entries containing "임시", "테스트 중", "나중에 삭제", "temporary", "test run", "delete later" → remove entirely

Algorithm

**CRITICAL — STRICT CHAIN ORDER: Steps 2→3→4→5 MUST execute in sequence. NEVER skip a level.**

Each step feeds the next. Root reads from monthly. Monthly reads from weekly. Weekly reads from daily. If you skip a level, the chain breaks and data is lost or corrupted.

Raw → [Step 2] → Daily → [Step 3] → Weekly → [Step 4] → Monthly → [Step 5] → Root
         ↑                    ↑                     ↑                    ↑
     reads raw           reads daily            reads weekly        reads monthly
     writes daily/       writes weekly/         writes monthly/     writes ROOT.md

**NEVER:**

• Modify ROOT.md based on daily or weekly data (root reads ONLY from monthly)
• Modify monthly based on daily data (monthly reads ONLY from weekly)
• Skip Step 2 or 3 because "there's nothing new" — always verify by checking files
• Touch ROOT.md directly without going through the full chain

Step 0: Pre-Compaction Snapshot

Before starting the compaction chain, preserve current working state:

1. Read `WORKING.md` 2. If it contains active task content (not just the empty template):

• Append a snapshot to today's daily log (`memory/YYYY-MM-DD.md`):

     ## Working State Snapshot [project]
     - context: pre-compaction automatic snapshot
     - state: [copy WORKING.md content]

• This ensures in-progress work is captured before any context compression

3. If WORKING.md is empty or contains only the template, skip this step

Step 1: Discover Candidates

Scan `memory/` for raw files. Group by date, ISO week, and month. Check each group against trigger conditions.

Step 2: Daily Compaction (max 1 per cycle)

**Input:** raw files (`memory/YYYY-MM-DD.md`) **Output:** daily nodes (`memory/daily/YYYY-MM-DD.md`)

For each date where raw exists and daily needs create/update:

1. Read raw file `memory/YYYY-MM-DD.md` 2. Count lines — compare against ~200 line threshold 3. Below threshold: copy raw verbatim to `memory/daily/YYYY-MM-DD.md` 4. Above threshold: generate keyword-dense summary 5. Write with frontmatter:

---
type: daily
status: tentative
period: YYYY-MM-DD
source-files: [memory/YYYY-MM-DD.md]
topics: [keyword1, keyword2, keyword3]
---

## Topics
## Key Decisions
## Tasks Completed
## Lessons Learned
## Open Items

6. If date has changed (raw is from a past date): set `status: fixed`

**Secret scanning:** The mechanical compacti