/codebase-index

name: codebase-index
description: 3-layer codebase indexing system for token-efficient code navigation. Builds compact index + symbol map so agents read targeted lines instead of entire files. Achieves ~5-10x token savings.

Codebase Index - 3-Layer Memory Optimizer

Build and maintain a progressive disclosure index that prevents agents from reading entire files when they only need a function signature or a specific line range.

When to Activate

• During `/crew init` (full build)
• During `/crew reindex` (full rebuild)
• When an agent needs to find code but the index exists (consult mode)
• After file edits (incremental update mode via index-updater skill)

Three-Layer Architecture

Layer 1 - Compact Index (`.claude/crew-index.json`, ~2-5KB)

The cheapest layer. Costs ~50 tokens to read. Contains just enough to route queries to the right file:

**Per-file entry:**

• `hash`: SHA-256 of file content (for change detection)
• `size`: file size in bytes
• `type`: language (javascript, python, typescript, etc.)
• `exports`: exported functions/classes/constants
• `imports`: external dependencies
• `functions`: all function/method names
• `classes`: all class names
• `lines`: total line count
• `category`: core | test | config | docs | util
• `summary`: 1-line description of what this file does

**Global metadata:**

• `version`: schema version
• `generated`: ISO timestamp
• `contentHash`: hash of all file hashes combined
• `dependencies`: package dependencies with versions
• `architecture.entryPoint`: main entry file
• `architecture.pipeline`: data flow description
• `architecture.patterns`: detected patterns (MVC, pipeline, event-driven, etc.)
• `stats`: totalFiles, totalLines, languages, lastIndexed

Layer 2 - Symbol Map (`.claude/crew-symbols.json`, ~5-15KB)

Mid-detail layer. Costs ~100-200 tokens for relevant section. Contains function signatures and relationships:

**Per-symbol entry** (keyed as `filepath::symbolName`):

• `type`: function | class | method | constant | type
• `signature`: full signature string
• `params`: parameter list with types if available
• `returns`: return type if available
• `calls`: what this symbol calls
• `calledBy`: what calls this symbol
• `lineRange`: [startLine, endLine]
• `description`: 1-line description

**Relationship maps:**

• `callGraph`: which functions call which
• `fileRelationships`: import/export connections between files

Layer 3 - Full Details (on-demand file reads)

Agents read actual file contents ONLY after consulting Layer 1+2 to know exactly which file and line range they need. Use the Read tool with `offset` and `limit` parameters to read specific line ranges.

Building the Index (Full Build)

When triggered for full build:

Step 1: Discover source files

Use Glob to find all source files:
- **/*.{js,ts,jsx,tsx,py,go,rs,java,kt,rb,php,c,cpp,h,cs}
- Exclude: node_modules, .git, dist, build, __pycache__, .next, vendor
- Also find: package.json, requirements.txt, go.mod, Cargo.toml, pyproject.toml

Step 2: Read each file and extract metadata

For each source file: 1. Read the file content 2. Compute SHA-256 hash (use `shasum -a 256` via Bash) 3. Extract exports (look for `module.exports`, `export`, `def`, `func`, `fn`, `public`) 4. Extract imports (look for `require`, `import`, `from`, `use`) 5. Extract function names and signatures with line ranges 6. Extract class names 7. Count lines 8. Categorize: core (src/), test (test/), config (root configs), docs, util (lib/, utils/) 9. Generate 1-line summary

Step 3: Build call graph

From the imports/exports and function calls data: 1. Map which files import from which 2. Map which functions call which (by scanning function bodies for calls to known symbols) 3. Identify entry points (files not imported by anything)

Step 4: Write index files

1. Write `.claude/crew-index.json` (Layer 1) 2. Write `.claude/crew-symbols.json` (Layer 2) 3. Report stats: files indexed, total lines, languages detected

Incremental Update

When a single file changes: 1. Recompute its hash 2. Compare with stored hash in crew-index.json 3. If different: re-read the file, update its Layer 1 + Layer 2 entries 4. Update the global contentHash 5. Do NOT rebuild entries for unchanged files

How Agents Should Use the Index

INDEX-FIRST PROTOCOL (mandatory):

Before reading ANY source file:
1. Read .claude/crew-index.json → find which file(s) are relevant
   - Match by: function names, exports, summary keywords, category
2. Read .claude/crew-symbols.json → find exact symbol and line range
   - Match by: symbol name, signature, calledBy/calls relationships
3. Read ONLY the specific line range you need from the actual file
   - Use: Read tool with offset=startLine and limit=(endLine-startLine+1)

NEVER:
- Read an entire file when you only need one function
- Grep the whole codebase when the index has the answer
- Skip the index because "it's faster to just read the file"

ALWAYS:
- Consult Layer 1 first (cheapest)
- Consult Layer 2 only if Layer 1 isn't specific enough
- Read Layer 3 (actual file) only for the exact lines you need
- After making changes, note which files were modified for index update

Token Savings

| Approach | Tokens Used | When | |----------|-------------|------| | Read all files | ~2000-10000+ | Without index | | Layer 1 only | ~50-100 | Finding which file has a feature | | Layer 1 + Layer 2 | ~150-300 | Finding a specific function | | Layer 1 + 2 + targeted read | ~300-500 | Reading + editing a function | | **Savings** | **5-10x** | Per agent interaction |

Edge Cases

• **New file not in index**: If an agent creates a new file, add it to the index immediately
• **Deleted file still in index**: On reindex, remove entries for files that no longer exist
• **Binary/large files**: Skip files > 100KB or binary files (images, compiled assets)
• **Generated files**: Skip dist/, build/, .next/ directories
• **Config files**: Index but categorize a